Online help TestBench Cloud Services

Online Help

TestBench Cloud Services, Version: 1.4

The TestBench Cloud Services provide functionality to support testing in software development projects.

Login

The system hosts many tenants, and for each tenant the users must be logged in separately to get access to their tenants' data.

For this, the users get login credentials from the their Tenant Administrator.


login screen
Login screen.


In addition to user login and password, the tenant name is required.
With the link "forgot your password?" a new password can be requested. Therefor the fields Tenant and the User login mus be entered correctly.

Reset password.
Reset password.

An new one time password will be generated and sent to the user.


login loading indicator

While waiting to login the loading indicator appears.

Home Screen


The home screen of the system shows the current products under test.


The home screen lists all products under test.
Home screen with two products like a Tenant Administrator sees it. A normal user will not see the second button.

A product is the container of all epics, (free) user stories, (free) test cases, test executions and defects are needed to evaluate the quality of the product.

Each product is listed with many additional information:
  • Nearby the name of the product the short view of the quality metric of the product is displayed. A click on it leads to the detailed metric.
  • The six most urgent elements are listed in the sequence of their urgency.
  • A little statistic which lists all elements with their activities urgency.

  • A click on the title of a product opens the product screen. A click on one of the elements listed, is a link to this element and opens this element directly.

    Product

    The add button creates a new and empty product. Only Tenant Administrators have the rights to do this.
    New products are created with the add button.
    Add button.

    To add a new product the name of the product must be entered.
    Add a new product with the name "My new product".

    Only a tenant administrator has the privileges to create, administer and configure a product.
    After creating a new product the product has only a name.

    new, but empty product
    New, but empty product.

    Only the Tenant Administrator creating the product has privileges to work within the product.
    Therefore some settings should be done. The settings of the product can be opened via the settings icon.

    settings icon
    Settings icon.

    At first the product should get a more detailed description. The description is set in the product's MASTER DATA.

    product description
    The Product prefix is automatically assigned to the product when it is created and displayed in MASTER DATA.

    The Timezone can be updated in MASTER DATA.Only users with roles admin and manager can change the timezones and all others users can just read it. It can be customized
    by selecting GMT Offset.
    timezone GMT Offset
    GMT Offset Timezone.

    The description is set in the MASTER DATA.

    The PRODUCT QUALITY is described in the Product Quality chapter.

    The next more important data to be set are the users, who shall have access to the product.
    The users must be created by a tenant administrator before they can be assigned to a product (see chapter User Management ).
    This users are assigned and configured in the product's USERS section.

    users section of the product
    The USERS section.

    With Add user an existing user can be assigned to the product. The shown users list to select an entry is reduced to the matching entries when there are some characters typed in.

    select an user from the list
    Assign a user to the product.

    The assigned users are listed in the users list of the product. There they get the intended rights.


    added user in the users list
    The user is added to the list.


    Inside a product a user can get the roles Test Manager, Test Analyst and Tester with the according rights:

    Element Right Test Manager Test Analyst Tester
    Users To change own password yes yes yes

    To change own user data yes yes yes
    Products To see the list of products with a role in it yes yes yes

    To delete a product yes


    To change the master data of a product yes


    To see the user list of a product yes


    To change the user list of a product yes


    To change the role of a user of a product yes


    To see the detailed quality metric of a product yes


    To see the list of responsible users for an element yes yes yes

    To read the product details yes yes yes
    Epics To create, to change, to delete an epic yes yes

    To be a responsible user of an epic yes yes

    To read details of an epic yes yes
    User Stories To create, to change, to delete a user story yes yes

    To be a responsible user of a user story yes yes

    To read details of a user story yes yes yes
    Test Cases To create, to change, to delete a test case yes yes

    To be a responsible user of a test case
    yes yes

    To read details of a test case yes yes yes

    To insert a test step in a test step block yes yes

    To change a test step yes yes

    To mark a test step block as empty yes yes

    To set or to remove the link of a precondition to a test case yes yes

    To mark the list of precondition as empty yes yes

    To create, to change, to delete a precondition yes yes

    To read the list of preconditions yes yes yes

    To change name of a precondition yes yes

    To change the status of a precondition yes yes yes
    Executions To start an execution yes
    yes

    To read details of an execution yes
    yes

    To set the status of an execution yes
    yes

    To set the execution result of a test step yes
    yes

    To change the responsible users of an execution yes
    yes
    Defects To create, to change, to delete a defect yes
    yes

    To read the list of defects yes yes yes

    To link a defect to an execution or to a test step yes
    yes

    To remove the link of a defect from an execution or from a test step yes
    yes
    Table of user rights by user role.


    user gets the test manager role
    The user gets the test manager role.


    The user gets full access when all the roles are set.
    When the user gets all three roles he/she gets full access.

    Product Overview


    In the product overview all the elements of a product are listed.

    It shows all elements of the types epic, user story, test case and defect independently of the hierarchy between the elements.

    The product overview lists all elements of the product.
    Product overview.

    In the following all the elements of a product are presented. The elements are always sorted by their urgency. The most urgent elements are located in the upper left corner. The less urgent elements a located in the lower right corner.

    The header of the product overview shows the title of the product, the button for filtering the elements in the screen and the button to open the products settings.

    With the filter the elements shown in the product overview can be filtered.
    the Button opens the filter:


    Open filter
    Open filter.

    With each character typed into the filter field, the name of the elements in the product overview are searched for matches. The contents is adapted immediately. The filter shows how many entries are found over all entries.
    The filter value will also be mirrored in the browser url, so it can be stored as a bookmark for later use. If a url with a filter is opened, the view will show only the matching elements.


    filtered contend of the product overview

    Product overview filtered by the string "Price".


    The filter is closed by clicking the filter icon again. This automatically deactivates the filtering, too. The click on the 'x' button clears the filter string.




    In the product home the user can navigate to elemets by clicking on pills.


    Pills

    The pills.



    Clicking on the user stories pill sets the filter by type to "User Stories" and filter by relationship to "Children of " the epic parent.


    navigate to user stories with pill


    User stories for epic "Price calculation".

    Clicking on the test cases pill sets the filter by type to "Test cases" and filter by relationship to "Children of " the user story or epic parent

    Clicking on the defects pill sets the filter by type to "Defects" and filter by relationship to "Children of " the test case parent


    navigate to test cases with pill


    Test cases for epic "Price calculation".


    The user can access each product or product element by using the TBID.

    The TBID is unique for all elements of a single product and has the structure PPP-XXX, where PPP is the product prefix and XXX is the product element suffix.

    Each product has initially the prefix TB‹id›, where ‹id› is the generated identifier of the product. For example 'TB15' for the product with identifier 15.


    This prefix is provided in the tab-card MASTER DATA below the title of each product, only the Tenant administrator can update it.


    product prefix
    Product prefix.


    Each product element has an initial TBID suffix 'XXX' as follows:

    E‹id›:, where ‹id> is the identifier of the epic.
    U‹id› :, where ‹id› is the identifier of the user story.
    T‹id›:, where ‹id› is the identifier of the test case.
    D‹id› :, where ‹id› is the identifier of the defect.
    T‹tcId›X‹execId› :, where ‹tcId› is the identifier of the test case and ‹execId› is the identifier of the execution.

    The suffix is provided in the detail of each element below the title.


    suffix of epic
    Suffix of an epic.



    By clicking on the icon , the link SERVER/TBID of the product element will be copied to the clipboard.

    Using this link in the browser address bar changes the address in the classic style SERVER/products/1/epics/3 and opens the element in detail view.


    Filtering


    It can filter items by : Field, Relationship, Type and/or by Responsible User.

    It can filter by field using the options: Title, Description, Status or Custom Field.


    Filter list
    Filter by field.



    The filtering will be possible after typing a text in the text input and choosing one of the operators: equal, not equal, contains, not contains, empty, or not empty.


    operator filter view
    It can choose the operator.



    The operator list:

    ∋ - contains: item is available and contains the given non empty string as substring.

    ∌ - does not contain : item is available and does not contain the given non empty string as substring.

    = - is equal to: item is available and contains exactly the given non empty string.

    ≠ - is not equal to: item is available and does not contain exactly the given non empty string.

    ∅ - is empty: item is available and its value is empty.

    ? - is not empty : item is available and its value is not empty.

    By choosing a Title, there are four operators possible: contains, not contains, equals and not equals.

    By choosing the field Description or Custom Field, there are six operators possible: contains, not contains, equal, not equal, empty and not empty.

    By choosing status , there are two possible operators: equal and not equal with a list of status possible.


    It can filter by relationship: children of, parents of or siblings of.


    Filter by relationship
    It can choose the relationship.



    It can choose element type: epic, user story, test case or defect after adding a text input.


    choose the type
    It can choose the element type.



    The filter by relationship parents of:
    When applied on epics it returns an empty list.
    When applied on user stories it returns their parent epics.
    When applied on test cases it returns their parents user stories and epics.
    When applied on defects it returns the test cases where these defects are assigned and the parent of those test cases.

    The filter by relationship children of:
    when applied on epics it returns their children user stories, test cases and their assigned defects.
    When applied on user stories it returns their children test cases and their assigned defects.
    When applied on test cases it returns their assigned defects.
    When applied on defects it returns an empty list.

    The filter by relationship siblings of:
    when applied on epics it returns all epics whose titles match the given string .
    When applied on user stories it returns all user stories under the same epic or all free user stories if the fetched ones are free.
    When applied on test cases it returns all test cases under the same user stories or all free test cases if the fetched ones are free.
    When applied on defects it returns the defects under the same test cases.


    Filter view by type
    It can filter by Type.


    Epics, User stories, Test cases, Defects : These filters works in groups (two or more filters can be selected at the same time), if a filter is selected the union is returned in relation to these filters.


    responsible filter view
    It can filter by responsible user.



    My items : Display only those elements for which the user is responsible for.

    Unassigned items : Display only those elements without responsible.

    And it can filter by users: Display elements related to the assigned responsible user.


    Example of filter
    Example of filter.


    Tree View


    On the left side of the product overview, a tree can be opened. This tree displays all the elements of the product in alphabetical order in case sensitive.
    The tenant administrator can not open the tree, only the users have at least one role within the product can open the tree view.


    Tree view filter
    Closed Tree View.



    The tree view displays the names of the Epics in the current product as top level nodes. There are also two sections for “Free user stories” and “Free test cases” below the list of Epics.


    The tree can be opened by clicking the icon.


    Tree view expanded
    Expanded Tree View.



    By selecting an item within the tree, my options should be displayed.
    There are three options: "Add", "Duplicate" and "Delete" an element.


    Tree view options
    The options should be visible.




    Under Epic, by clicking the (+) button a new Epic or User story can be added.

    Tree view options
    Add elements under Epic in tree view




    Under User story by clicking the same button, a new User story or Test case can be added.

    Tree view options
    Add elements under User story in tree view




    Under Test case only a new Test case can be added.

    Tree view options
    Add elements under test case in tree view




    By clicking the duplicate button, a new copy of the selected element will be created. The new element will have the name of the original element with prefix 'copy: '. The children will not be duplicated.

    Tree view options
    Duplicate elements within the tree view




    By clicking the delete button, the selected element will be removed from the tree as well as his children elements if they exist.

    Tree view options
    Delete elements within the tree view



    Under defect there is just the "Delete" option.
    Under free user story and free test case there is just the "Add" option.


    Within the tree view the user can move the test cases and the user stories by drag and drop them. When an element is dragged the possible locations for drop it within the tree will be marked.


    drag and drop
    Possible locations to drop user story .


    Drag and drop options:

    Free user stories and user stories can be dragged and then dropped only under Epics or Free user stories folder.
    Free test cases and test cases can be dragged and then dropped only under User stories, Free user stories or Free test cases folder.
    Epics, defects, free test cases Folder and free user stories folder can not be can be dragged and dropped.

    Epics


    Inside the product overview, with the add button new epics, free user stories (user stories without a link to an epic, see User Stories chapter)  and free test cases (test case without a link to a user story, see Test Cases chapter) can be created.

    An epic is a kind of high level requirement. It describes the functionality of a system on a high abstraction layer and cannot be realized without some kind of decomposition in smaller parts. These smaller parts are the user stories.


    Add epic, free user story or free test case
    Add epic, free user story or free test case.


    Add new epic
    Add a new epic.


    Right after creating a new Epic, by default, the focus is set into the description field.

    New epic created

    Below the epic title a link referring to the element with the TBID can be copied to the clipboard.


    testbench id

    Each field of the epic has a so called quick edit mode that changes only the desired field.

    quick edit mode
    Quick edit mode of the description field.

    To activate the quick edit mode for a field, the small edit button at the right top of the hover area of the field must be clicked. Additionally it also works to click on the gray area. A text can be selected without entering the quick edit mode. If a text is already selected, a click on the gray area will deselect the selected text. The click on the quick edit button always will enter the quick edit mode, even if a text is already selected.

    Each epic can get several responsible users. When responsible users shall be added, a list of all user of the product who have the correct rights is displayed to select a user. In the list, the name and the roles of the user is shown.

    add responsible user
    Add a responsible user from the users list.

    A responsible user is added
    A responsible user is added.
    Moving through the user chips can be done using keyboard interaction, the arrow keys and select/deselect them with the space. User chips also gain focus when clicked, ensuring keyboard navigation starts at the appropriate chip.

    Clicking on the 'X' symbol nearby a users name removes the user from the list.

    Deactivated users will be shown in light grey color, but cannot assigned as responsible user anymore.
    Deleted users are deleted from the users list but an entry "Deleted user" will be displayed instead to show that there were one or more deleted users in the list.
    When editing a users list with deactivated or deleted users, this users will be removed from the list when starting the edit mode.

    deactivated responsible user
    Deactivated responsible user.


    In order to delete epic, the condition is that epic should not have any User story. If there isn't any user story a confirmation modal will be open, asking `Are you sure you want to delete epic Epic 1`.


    open delete epic

    Delete epic modal.


    deleted responsible user
    Deleted responsible user.


    The deactivate or delete user see the User Management chapter.



    open activities in epic
    The Activities column is described in the Activities chapter.


    test cases in epic
    The Test Cases column is described in the  Test Cases Chapter and the Test Status chapter.

    preconditions in epic
    The list of Preconditions is described in the Preconditions chapter.

    User Stories


    A user story describes a requirement of a system in a detailed way. The user story can be implemented directly.

    Normally user stories are linked to an epic, which collects all user stories of the same topic. But in the product overview a so called free user story can be created, too. The only difference to normal user stories is the non existing link to an epic. Therefore free user stories can be accessed via the product overview. The following descriptions are valid for both variants of user stories, except the linking to an epic.


    A user story is created from the epics screen and therefore the user story is linked to an epic. For this the add button in the epic screen is used:


    Create an user story out of the epic screen.

    Create a user story.


    When a user story is created, the corresponding epic is shown at the left of the screen as a smaller card, so the link to the epic is always displayed.

    New epic with corresponding epic.
    New user story with the corresponding epic.

    The click on the epics card opens the epic and displays the user stories linked to the epic as smaller cards.

    Epic with user story.
    Epic with linked user story.

    To open a user story, the card of the user story can be clicked.


    The (free) user story has the same fields than an epic and can be edited in the same way (see chapter Epic).

    Test Cases


    Test Cases are the elements to contain the test specification. At the moment test cases only supports test cases for manual execution.
    Normally test cases are linked to user stories, but in the product overview a so called free test cases can be created, too.The only difference to normal test cases is the non existing link to a user story. Therefore free test cases can be accessed via the product overview and test cases can be accessed via the corresponding user story. The following descriptions are valid for both variants of test cases, except the linking to a user story.

    A test case is created from the user stories screen and therefore the test case is linked to a user story. For this the add button in the user stories screen is used in a similar way like in the screen an epic:


    Create a test case.

    Create a test case.

    The name, responsible user and the description are used the same way like in epics and user stories. But a test case has some more special sections with special fields.


    Screen of my first test case.
    A test case right after creation.

    Like a user story, the test cases are linked to their parent element. In the screen the whole trace form the corresponding epic and the user story is displayed.

    Trace form epic to test case.
    The trace from epic to test case is always displayed.

    open activities in test case
    The Activities column is described in the Activities chapter.

    start execution button
    The Start Execution button, starts e execution of the test case which is described in the Execution chapter.

    The Test Execution History section is described in the Test Execution History chapter.

    The Test Automation section is described in the Test Automation chapter.

    The Test Sequence section is described in the Test Sequence chapter.

    Test Sequence

    Each test sequence consists of the six sections Preconditions, Preparation, Navigation, Perform Test, Check Result and Reset Environment.

    When a section is fresh and empty the blue question mark is displayed: 

    Blue question mark signals an empty section.
    A blue question mark signals an empty section.

    All the sections are used the same way (except the Preconditions section ):


    A section can be edited separately via their quick edit mode.

    Quick edit mode hover of preparation section.
    Quick edit mode hover of the preparation section.

    Initiated edit mode of the preparation section.
    Initiated edit mode.

    A test step can be added via clicking on the plus button or by clicking in the field directly.
    First test step added to section preparation.

    Multiple preconditions or test steps can be created right after one-another by pressing CTRL + ENTER in the Input field.
    Pressing CTRL + ENTER when the input field is empty will finish that precondition or test step block, and open the next block underneath in edit mode.


    First test step added.

    Test steps can be deleted by the red minus button and can be moved via drag and drop by the double line button.

    First test step added to section preparation.
    First test step added.

    When a section has got one test step in minimum, the blue question mark of the section turns into a white ring.
    During the test execution, each test step can be set as passed or failed and can get links to defects.

    Example of a test sequence.
    Example of a test sequence.

    The intentions behind the different sections are:

    Preparation: Used to do some preparation before the test can be executed. Typical preparations are loading special data into the system under test or set some options needed.

    Navigation: By these steps, the tester is typically directed through the system under test to reach the location or the state, the test needs to start.

    Perform Test: These are test steps, which stimulates the system under test, e.g. by inputting some data or changing the state of a functionality.

    Check Result: These steps hold the expected results. When are all the expected results are fulfilled, normally the test case is passed.

    Reset Environment: After the execution of a test case, sometimes some cleanup is needed to get a proper environment for the next test cases.

    Preconditions

    The list of preconditions is the starting section of the test sequence. A precondition describes a kind of state, which must be fulfilled to be ready to test. A typical precondition is the existence of special test data, which is needed for the execution of the test case. Preconditions are not only used in a single test sequence, but they can describe conditions in much more broader scope. For this, preconditions can be used all over a product.

    Preconditions are always created and selected in the precondition sections of test cases.

    Precondition section with one entry.
    Precondition section with one entry.

    Because of the product wide scope, the precondition must not be copied when they shall be used more than once. As soon as there are three characters typed into the field the system displays the list of matching preconditions. At first the complete matching entries, then the entries only matching a part of the input.

    All preconditions matching the typed in word.
    All preconditions matching the typed in word "the".

    The entered preconditions are not only displayed in their test cases, but on the level of user stories and epics, too. The list of preconditions on epic level and user story level contains all preconditions used in the according test cases.


    As soon as in minimum one precondition list has got an entry, the blue question mark turns into a red filled circle with a 0% caption. This means a fulfillment degree of the entries in the list of 0%.

    Fulfillment degree of zero for the preconditions.
    No precondition is fulfilled.


    Each fulfilled precondition increases this degree. Preconditions can be marked as fulfilled without starting any edit mode or an execution.

    One of two preconditions is fulfilled.
    One of two preconditions is fulfilled.

    All four preconditions are fulfilled.
    All four preconditions are fulfilled.

    The preconditions can be marked as fulfilled in the lists of all three levels. The marking of a precondition as immediately populated to all other elements, the precondition is listed.

    Executions

    For the test cases there is a special view for the test execution, called execution. This view is initiated when clicking on the Start Execution button in test case.
    start execution button.

    When the test execution is started, the test case is displayed in the execution view.

    Execution view of a test case.
    Execution view of a test case.

    In the execution, responsible users with the role tester can be added.
    The test result status can be set, but by default this status is evaluated automatically from the test results of the test steps of the test sequence. When there are no test steps, the status mus be set manually. When the status is set manually, a little wrench is displayed near by the status.

    The preconditions show their fulfillment status, but cannot be change (e.g. this can be done from the test case view).

    Each test step can be marked as passed via the circle with the hook, and marked as failed via the circle with the bug inside.

    Test steps marked as passed or failed.
    Test steps marked as passed or failed.

    The test step status can be toogled by multiple clicking these buttons.

    Clicking the bug icon on global defects section or on a test step, opens the form for creating a defect. The placeholder of the form specifies from where the defect is being created.

    Add global defect
    Add global defect form.

    Add test step defect
    Add test step defect form.

    When creating a defect by clicking directly on the add button will create a global defect.
    Add defect button.
    For details see the chapter Defects.

    If in test sequence a test block is empty, it has no test steps, then that test block does not appear on execution view. If all the test blocks or preconditions are empty none of them appears on execution view.
    Execution view of a test case without test blocks.

    In the Global defect(s) section, the user can create defects which cannot assigned to a single test step but relates to the whole test case.

    Stop execution button.

    With the STOP EXECUTION button, the execution of the test case is stopped, but not aborted. The execution can continued with the help of the Test Execution History.

    Test Status

    The status of a test case is displayed in the top most section of the test case screen.

    Test case with status In Progress.
    Test case with status In Progress.

    The status of a test case can be:


    Status Description
    Ready
    The test case is ready to test.
    In Progress
    The test execution of the test case is running.
    Passed
    The result of the last execution is passed.
    Failed
    The result of the last execution is failed.


    The status is always displayed as a colored pill. These pills are used to summarize the status information on the level of epics and user stories, too.


    A user story card with all the four status pills.
    All the four status pills.

    The number in front of the status is the number of test cases with this status, which belong to the epic or user story.


    Test Execution History

    The Test Execution History lists all performed test runs and the entries in the list are the links to the detailed execution data of each test run.

    Three entries in the test execution history.
    Three entries in the test execution history.


    Each entry in the history shows for this execution the date, the user who has performed it, the status and the number of defects linked to this execution. The latest execution is listed as first entry and this latest execution determines the current result of the execution.

    With a click on an entry in the list, the according execution is opened and the user can change any data in the execution.

    Test Automation

    Each element of type test case contains a section for test automation.

    It can be used to link test cases to test automation tools and frameworks (e.g. RobotFramework) and to reflect results of automated test cases in TestBench CS.


    Test automation block.


    The link can be established using one of the following options:

    1 The user plans automation of test cases which are defined within TestBench CS. To do this, the user sets the toggle switch to "on".

    Test automation toggler.

    After that the user provides an External ID. The same External ID needs to be referenced from the test automation framework to identify and connect to the given test case.

    Test automation external id.

    2 Test cases may be merely implemented in a test automation framework, without being reflected by a test case within TestBench CS.

    Using the TestBench CS API, corresponding test cases can be easily created within TestBench CS.
    In this case, the "Automated" flag is toggled on by the API call, and the external ID is also set via the API.



    Note:

    By default, TestBench CS asks the user to review each change made by the test-automation framework via the API.
    If the review was done, click the check box.

    Test automation, review confirmation.
    Test automation green chek box.

    Defects

    When a test step is marked as failed or the defect button in the Global defect(s) section is clicked, the screen to create a new defect is opened.


    Add a new defect.
    Add a new defect.

    Entering the name of the new defect, creates the defect and opens the defect page where all the details of the failure can be added.

    Enter detailed defect data.
    Enter detailed defect data.

    In addition to the name and description, responsible users can be added and the status, priority and severity of the defect can be set.

    Different defect status.
    Different defect status.

    The starting status is NEW. In the quality metric of the product, the status CLOSED regards the according defect as done.

    Different defect priority.
    The priority can be set to Low, Medium and High. The default priority is Medium.

    Different defect severity.
    The severity can be set to Minor, Moderate, Major and Critical. The default is Moderate.

    Activities

    One of the most important elements in the TestBench CS are the activities. An activity is work to be done to finish one of the elements inside a product. The activities are displayed in a column in the header section of epics, user stories and test cases and in the small cards of epics, user stories and test cases.


    Three open activities.
    Three open activities for this element.

    Activities are set automatically for epics, user stories, test cases and defects.

    Element Activity
    Epic
    To fill the description.

    To create at least one user story related to this epic.
    User Story To fill the description.

    To create at least one test case related to this user story.
    Test Case To fill the description.

    To create at least one precondition or
    to mark the preconditions section as intentionally left blank.

    To create at least one test step for each section in the test sequence or
    to mark the sections as intentionally left blank.

    To execute the test case.
    Defect To close resolved defects.

    Table of the activities.

    When an activity is done the according activities counter is decreased automatically. When the counter is set to zero the activities icon is not displayed.

    To indicate the urgency of an activity, there are three severities each activity can get.

    Severity Description
    Info
    The activity is set less than seven days ago.
    Flash
    The activity is set more than six days ago but less than fourteen days ago.
    Flame
    The activity is set more than thirteen days ago.

    Table of activity severities.

    The three states are marked with icons and colors.

    Activities in all the three states.
    Open activities in all the three severities.

    For the state Flame on the right side of the small card of the according element, a red bar is shown.

    Small card of a flamed epic.
    Small card of a flamed epic.

    To know exactly which activities are open, the icons in the activities section can be clicked. This displays the list of activities in order of their severity:


    list of open activities
    List of open activities of an epic.

    In the list each entry shows the missing activity in the first line and the icon and name of the affected element.

    When the current element is affected, the second line is not displayed:

    activities list of the current element
    List of activities of the displayed user story.

    A click on an entry of the list jumps to the affected element.

    Custom Fields

    A very flexible and important element in TestBench CS is the ability to define and use custom fields.


    A custom field is a field that is not predefined by the developers of TestBench CS. It can be defined by the users that have the Tenant Administrator role.

    Custom fields can for example be used to define a set of additional information which may be needed in one or more of the elements that can be used in the TestBench CS. For example epic, user story and all other elements.

    If a serial number that must be entered in each of your test cases has to be used there is no need to use, for example the description field, but it is just a matter of adding a custom field, give it a name like "SerialNumber", add a label for that field, may be "Serial Number” and define that this field will be used for all test cases.
    After that is done this field can be used in all of your test cases.


    Configuring Custom Fields

    The first thing is to open the dialog where custom fields can be managed. The navigation can be found here if the user has the role of a Tenant Administrator


    This is where navigation to custom field management happens
    Menu entry to the custom field
    management for tenant administrators.

    A modal dialog opens. Everything that needs to be done to add or change a custom field or custom field block can be done here.

    The modal contains three different tabs:

    1. CUSTOM FIELDS
    2. CUSTOM FIELD BLOCKS
    3. CUSTOM FIELD BLOCK ASSIGNMENT


    Custom Field Management.

    CUSTOM FIELDS

    The custom fields window is split in two parts.
    The left part gives an overview about the custom fields already defined and to enable the user to add a new custom field. It is also used to select custom fields that need to be modified

    On top of the left part of the window there is a head line. In this head line the user is able to add a new custom field.

    click this button to add another custom field

    active add another custom field entry point

    With the trash can one or more custom fields can be deleted, multiple selection of the fields is possible.


    delete the selected custom field

    ATTENTION: Only unassigned custom fields can be deleted

    By clicking the trash can icon a dialog opens that informs the user which of the selected custom fields can be deleted.

    delete the selected custom field dialog

    Below the head line there is a table, which shows a list of all already defined custom fields. This list can be sorted by Name, Label, Type and Block Assignment.


    list of all custom fields


    The right part of the window is dedicated to give access to the details of one single custom field that is selected on the left side. It is also used to enter all new values needed if a new custom field is added.


    custom field right column


    Name
    Defines the name that this custom field has internally in the application.
    Label
    Defines the label, which is used when this custom field is displayed. After creating a custom field per default the label gets the same value like the name.
    Default Value
    A default value for the field can be added. This default value is always displayed as long as the field contents is not changed.
    Custom Field Type
    The Custom Field Type is used to define the type of the custom field. Available types are: SingleLineText, MultiLineText, Link, Attachment, Toggle.

    ATTENTION: Changing the type subsequently, ALL saved values will be lost in any element as soon as you have selected and changed the type.
    Block Assignment
    Block Assignment is used to place the custom field in one or more custom field blocks. Available custom field blocks are defined in the CUSTOM FIELD BLOCKS tab.

    Add block assignment for a selected custom field

    Remove block assignment for a selected custom field

    The block assignments of the selected custom field can be removed, by clicking the red round button on the right of the custom field block name. A dialog appears that asks if the user is sure that he wants to remove the assigned custom field block.

    delete block assignment of custom field

    remove assigned custom field block




    Current custom field limitations are:

    Type Maximum text length Line breaks allowed Trimmed
    SingleLineText
    255 printable characters No No
    MultiLineText 65 535 characters Yes No
    Link 2000 characters No No

    SingleLineText

    single line custom field in show mode

    An empty single line text field in display mode.


    single line custom field in edit mode
    A single line text field in edit mode could be filled with 255 printable characters.

    MultiLineText (with markdown-support)

    multi line custom field in edit mode

    A multi line text field in edit mode.
    multi line custom field in show mode

    A filled multi line text field in display mode.


    The field can be filled with multiple lines until the field limit has been reached.
    In this field you can use also markdown-syntax which will then be rendered accordingly. More information regarding markdown can be found at: Markdown-Cheatsheet

    Link

    Link fields are a little bit different from other fields, especially from single line text and multi line text fields.
    They should contain a link like "protocol://path". At the left side of the link field, there is an icon. Via click on a green icon the entered link opens in a new tab. Just clicking on the link itself inside the field will not result in opening this link, but in starting the edit mode.
    Note that file links like "file://path" are blocked by most browsers.


    link custom field in show mode
    The icon is displayed in green if a link field is filled. Be aware that this view is not a check for validity or availability of the link.


    link custom field in edit mode
    A link field in edit mode. In this mode the icon is grey.


    Attachment

    The Custom Field for attachments contains a file drop zone, which allows to upload one or more files.


    Custom field Attachment

    The list of uploaded files will be shown below the file drop zone.


    c

    All custom fields could be filled with data by clicking on the green edit icon or into the grey area. Both will be shown if the mouse hovers over the field.
    In edit mode the data will be saved by clicking the hook icon. To discard the last input the "x" icon must be clicked.
    As long as a custom field is empty "Add 'Label of the field' " is always displayed instead of the content.


    Toggle

    The Custom Field of type toggle contains a not editable text and a button of type toggle, which can be set to checked or unchecked


    custom field of type toggle


    CUSTOM FIELD BLOCKS

    The custom field block window is split in two parts.
    The left part gives an overview about the custom field blocks already defined and enables the user to add a new custom field block. It is also used to select custom fields blocks that need to be modified.

    On top of the left part of the window there is a head line identical like in the Custom Field tab. It allows the user to add a new custom field block.

    Below the head line there is a table, which shows a list of all already defined custom field blocks. This list can be sorted by Name and Product Assignment.

    Product assignment column displays the products in which the custom field block is assigned.


    list of all custom field blocks


    The right part of the window is dedicated to give access to the details of one single custom field block that is selected on the left side. It is also used to enter all new values needed if a new custom field block is added.

    right column of custom field blocks tab

    Name
    Defines the name that this custom field block has internally in the application.
    Product Assignment
    Defines the product assignment of the selected custom field block, which by default is 'All Products'. The product assignment of a custom field block can be changed by selecting the wanted product in the list of available products, which appears by clicking on 'Assign products'.

    custom field block product assignment

    Custom Field Assignment
    Custom Field Assignment is used to assign custom fields to the selected custom field block. Available custom fields are defined in the CUSTOM FIELDS tab.

    Add block assignment for a selected custom field

    Removing a custom field assignment from the selected custom field block can be done by clicking the red round button on the right side of the row, which opens a dialog that asks if the user is sure that this assigned custom field wants to removed.

    remove assigned custom field

    An other possibility is changing the custom fields order in the selected custom field block, and this is done by drag and dropping the custom field inside the custom field assignment table, by the icon of two horizontal lines on the right side of the row. drag and drop custom field assignment


    CUSTOM FIELD BLOCK ASSIGNMENT

    The custom field block assignment contains the available containers of the application. Each container has predefined anchors.

    Container Anchors
    Epic
    Before Description

    Before Preconditions

    Bottom
    User Story Before Description

    Before Preconditions

    Bottom
    Test Case Before Description

    Before Execution History

    Before Test Sequence

    Bottom
    Test Execution Before Description

    Before Execution History

    Bottom
    Defect Before Description

    Bottom



    custom field block assignment container


    On the anchors of one container the user can assign the available custom field blocks.
    custom field block assignment anchor

    ATTENTION:
    1. The same custom field block can not be assigned twice in an anchor.
    2. If an anchor has already assigned custom field blocks, the user can not assign new custom field blocks that contain custom fields that are already assigned to other custom field blocks in that anchor.


    The order of the custom field blocks inside a anchor can change by drag and dropping the custom field block, using the two horizontal line icon on the right corner of the custom field block row. The drag and drop option can be used to move the custom field block from an anchor to another and from a container to another.



    ATTENTION:
    A custom field block can not be moved from a container to another if the other container already contains that block in one
    of its anchors.

    drag and drop custom field block in container
    A custom field block can be unassigned from a anchor, respectively one container, by clicking the red round button on the right corner of the custom field block row, which opens a dialog that asks if the user is sure about removing the assigned custom field block.

    remove assigned custom field block from container

    Usage of Custom Fields

    As previously explained custom fields can be used in all elements that are at hand for the user of TestBench CS.

    The custom fields are displayed in all elements where the user assigned them.
    The order of the custom field blocks in a container and the custom fields inside a block appear exactly like the user defined them in Custom Field Management

    Here a sample for a test case:

    test case custom field blocks

    In the TestBench CS there is one place where the user will see two different custom field parts. Whenever a new test execution is created, the custom fields for that test execution will be available. As the application saves the values of different fields from the test case which is the source for that test execution, the application saves the labels and values for the custom fields defined for that test case.
    All those so called frozen fields will be shown in the last section of that execution named "<block name> (specification)" and they are not changeable regarding custom field type, label or content.

    test execution sample



    Progress Indicator

    Another mechanism to display the progress of an element is the progress indicator. Each epic, user story and test case have a special progress bar at the left side of their view or small card.

    Progress indicator of a test case.
    Progress indicator of the small card of a test case.

    The indicator has two bars:

    For the epics and the user stories the indicator displays sum of the progress of all according sub elements.

    By default the system assumes that 100% test specification progress can be reached with at least five sub elements. This means that a user story must have at least five test cases with 100% test specification progress to reach the 100% test specification progress for the user story. The same goes for an epic.

    For test cases the progress indicator changes its behavior when the test case has one of the status Ready, Passed or Failed. In this cases the indicator has a single bar filled with the color of the according status pill.

    Indicator for status Passed of a test case.
    Indicator for status Passed of a test case.


    Epic with its user stories and progress indicators, activities and execution status.
    Epic with its user stories and progress indicators, activities and execution status.

    The small cards of the user stories are listed in the sequence of their urgency. The most urgent user stories are listed at the top.

    While hovering over the specification and precondition progress bar a tooltip shows the progress percentage.
    Specification progress tooltip. Preconditions progress tooltip.

    Product Quality

    The progress of the quality of a product is measured with the help of the quality metric. The metric is displayed in the QUALITY METRIC section inside the settings of a product.

    As specific goal to be reached quality goals can be set. The Overall Quality Goal is evaluated from several Individual goals.

    List of quality goals of a product.
    List of quality goals of a product.

    Each half star of an individual goal counts 10% and each full star counts 20%. With clicking on the stars the stars can be set or reset.
    Annotation: To reach a full star, the click must be located in the upper right corner of the star.

    The average of all individual goals is set as Overall Quality Goal. In the example above the overall quality goal is 83%.

    Detailed quality metric of a product.
    Detailed quality metric of a product.

    At the left side of the metric the Overall Quality Goal is marked with a flagged green line. The current degree of quality is displayed with a thick green line.
    At the right side the open defects are counted and the highest number and the current number of defects are displayed. The current number is only displayed when this number is lower than the highest number. The shown period of the graph is 31 days back.

    A mouse over effect displays for each day the detailed data.

    The quality goal is increased by executions of test cases with the result Passed.

    The goals are to reach the Overall Quality Goal and to decrease the number of open defects. A defect must be set to the status CLOSED to decrease the number of open defects.

    A smaller and shorter version of the quality metrics is displayed in the product title.

    Small quality metric.
    Small quality metric.

    The small metric only displays the last seven days and has no mouse over effects. With a click on the small metric, the detailed metric is opened.

    User Management

    The tenant administrators must create accounts to grant access to other users.

    Menu entry to the user management for tenant administrators.
    Menu entry to the user management for tenant administrators.


    User list of a tenant.
    User list of a tenant.

    With the help of the ADD button a new user can be created.

    The data of a new user is entered.
    The data of a new user is entered.

    When a new user is created the tenant administrator assigns a one time password to the user and the user can only get access when she or he set a new password during her/his first login.
    A confirmation email is sent to the entered email address. As long as the user does not confirm the email address with the help of the included link, the user get no access to the system and in the detailed data of the user the email address is marked as pending. The confirmation must be done after 48 hours.

    Detailed data of a new user with an unconfirmed email address.
    Detailed data of a new user with a unconfirmed email address.

    In the users detailed view the users data can be changed by tenant administrators.
    When the email address is changed a security warning email is sent to the old email address and a confirmation email to the new email address.

    To change the password during next login can be forced by the according switch.

    In the same way the user can get rights inside of the product settings the rights can be granted in the user management.

    When selecting one or more user, in the Users table header two icons appear:

  • deactivate the selected user entries: deactivate user
  • delete the selected user entries: delete user

  • When a deactivated user is selected the deactivation icon toggles to the activation icon and the user can be activated:  activate user


    Deactivate the user Chuck Norris

    In the same we way can activate the deactivated user.


    Activate the user Chuck Norris

    Multiple users can be activated/deactivated at the same time, as long as all the selected users are activated or deactivated.


    Activate/Deactivate multiple users at the same time

    A user may not deactivate or delete himself. Therefore, the icons for these actions are not displayed when the currently logged-in user is selected.


    A deactivated user cannot login to the tenant and cannot be assigned as responsible person.


    Each user can change personal data by accessing My Settings found as menu entry over login name.

    Settings of the user Chuck Norris.
    Settings of the user Chuck Norris.

    When the user changes her/his own email address a security warning email is sent to the old email address and a confirmation email to the new email address.The change of the password must be authorized by the old password.

    Settings for changing password


    After switching to password view, there are three required fields. First one is old password (which will be validated if its correct only after clicking CHANGE PASSWORD button), and the two other fields are new password and confirm new password (which will be validated while user is typing).

    Translation

    Translation is a supportive feature that allows the user to translate text that is entered from a base language into different target languages.
    To use this feature it has to be activated in two steps.
    First one Tenant Administrator has to activate that translation can be used tenant wide.
    Second the user itself has to set its preferences.

    Tenant Administrator Settings to use translation service


    A user which is a Tenant Administrator, should configure the base translation option which are then available to all tenant users. He will do this in a translation dialog that can be opened from the Tenant menu.

    Tenant menu translation option

    A click on the Translation option at the tenant menu opens the Translation dialog.

    Translation Dialog
    On-the-Fly translation
    Enables or disables translation for all users in all products of this tenant.

    Enabled:
    Translation enabled
    Disabled:
    Translation disabled
    DeepL Authentication-Key
    TestBench CS is offering a trial for translation. This trial has a limit of 20 thousand characters which can be translated.
    When the trial has expired the tenant can buy his own license from DeepL and enter his own DeepL Key here.
    This license should be a DeepL Developer license. The DeepL License will come with additional costs.

    Translation authentication key
    Source Language
    The Tenant Administrator selects the base language that will be used for translation.
    The available languages now are:
    English, German, French, Italian, Dutch, Polish, Spanish, Portuguese, Russian and the Automatic option.
    The chosen source language by the Tenant Administrator, is the default source language for all the other users.

    Translation source language
    Target Language
    The Tenant Administrator chooses the target language into which the text should be translated.
    The available languages now are:
    English, German, French, Italian, Dutch, Polish, Spanish, Portuguese and Russian.
    The chosen target language by the Tenant Administrator, is the default target language for all the other users.

    Translation target language
    Characters Count The number of characters used compared to the character limit, which as trial is 20 thousand.

    Translation characters count


    Users settings to use translation service


    A user can change translation settings for himself as soon as translation was enabled by the Tenant Administrator.

    Translation tab on My Settings

    On-the-Fly translation
    The user can enable/disable translation for himself here.
    Source Language
    The user can choose the base language that will be used for translation.
    The available languages now are:
    English, German, French, Italian, Dutch, Polish, Spanish, Portuguese, Russian and the Automatic option.

    Target Language
    The user chooses the target language into which the text should be translated.
    The available languages now are:
    English, German, French, Italian, Dutch, Polish, Spanish, Portuguese and Russian.



    Using Translation Service


    When translation is enabled for the tenant and the user like described above, the user will see a flag which indicates that a translation is possible.
    When the user clicks the flag, translation of the text is done on the fly and the results are shown to the user.

    Translation flag icon

    Translated text

    File Upload

    Using the Custom Field of type Attachment you can upload and download files on an Epic, User Story, Test Case, Execution or Defect.
    Assigning a Custom Field of type Attachment on a existing Custom Field Block (i.e. on an Epic), will show a file drop zone which allows uploading files by either clicking on it or by dropping file(s) on the file drop zone.



    Custom field Attachment

    Clicking on the file drop zone will open a File Dialog (File Selector/Chooser) where you can select one or more files.
    You can upload files of any type, but restricted to a maximum size of 10 MB per file. The selected file(s) will be listed below the file drop zone.

    The file uploading progress is visualized on the right side of the file name.



    Uploaded File Progress

    The size in MB is displayed on the right side of each uploaded file. Below the uploaded files the total of the file sizes is shown.


    Uploaded Files

    Each uploaded file can have its own description which is shown only after selecting a single file.
    The description can be edited by clicking on the "Description" placeholder or on the green round button (Edit Button).


    Uploaded File Description

    The description is only displayed when selcting a single file.


    When selecting an uploaded file, the icons for Download and Delete appear.

    Download and Delete files options

    The Download option supports multiple file download of up to twenty files. This is achieved by selecting the respective files and clicking one of the Download icons.
    The selected files will be downloaded as a Zip file.


    When clicking the Delete icon, a Delete File Dialog opens, which allows to delete the selected file(s) or to cancel the process.


    Delete files dialog

    If the file upload fails, a Retry icon allows to re-initiate the upload.


    Retry upload file

    Requirements System

    TestBench CS allows to import epics and user stories from Jira to your product in TestBench CS. The imported requirements are read-only in TestBench CS, and Jira is the master of the data.


    During a synchronization with Jira, existing requirements are updated and new requirements are added in TestBench CS. Although the imported epics and user stories are read-only, they can be linked to user stories and test cases created in TestBench CS. Imports do not affect these user stories or test cases created in TestBench CS. That way, you can connect all test cases with the proper epics and user stories of your Jira project.


    Settings for connecting to Jira

    To import the epics and user stories from your Jira system to TestBench CS, the Jira URL (HTTPS only) and a valid login and password are required.


    If you are using Jira Software Cloud, you need to provide an API token, instead of the password. To get an API token please read the Jira Software Cloud help.


    The Jira Server Certificate field can be left empty in most cases. It is needed only if connecting to a self-hosted Jira server that uses a self-signed HTTPS certificate - in this case, enter here the Base64 encoded PEM format certificate (i.e., a certificate text starting with "-----BEGIN CERTIFICATE-----" and ending with "-----END CERTIFICATE-----").


    The epic configuration and user story configuration allows to specify the JQLs that control which issues will be imported as epics and as user stories.
    Note that the JQLs should not contain any "ORDER BY" clauses, because TestBench CS internally adds an "ORDER BY created ASC" clause to ensure optimum import results. During the configuration of the Jira connection and the JQL settings, TestBench CS provides feedback on how many elements the synchronization will import into TestBench CS.


    The user story configuration also allows to specify the Epic Link Path which is used to link a user story to an epic.

    Epic Link Path configuration cases:

    In Jira Cloud classical projects, the Epic Link Path is determined automatically.
    In Jira Cloud next generation projects, the Epic Link Path should be configured as "fields.parent.key".
    For English Jira Server installations, the Epic Link Path is determined automatically.
    For Jira Server installations in other languages, determine the Jira custom field id of the epic link (e.g., "customfields_10022") and configure the Epic Link Path accordingly as e.g. "fields.customfields_10022".

    User Story Configuration



    After configuring the jira, epic and user story configuration, you can start the import of epics and user stories from Jira into TestBench CS via the "SYNCHRONIZE NOW" button. Note that if issues are moved or deleted in Jira while a synchronization is running, in TestBench CS some of the imported requirements may not show the expected status. This inconsistency will mend automatically with the next synchronization.


    You can synchronize Jira without a valid license 10 times per product. Afterwards the "SYNCHRONIZE NOW" button will be disabled. To enable the full synchronize feature it is required to buy a license via the "SHOP"-button.


    You can view the Jira log by selecting "Log".


    Log for Jira connecting and synchronization


    You can download the Jira log via the "DOWNLOAD LOG" button. After pressing the button a "Save As"-dialog opens.


    System Requirements for importing requirements from Jira

    Requirements can be imported from Atlassian Jira Cloud instances.


    Requirements can also be imported from a self-hosted Jira server, provided a recent version of Jira Software Server is installed and the Jira server is reachable from the TestBench CS server via HTTPS.


    Tested with the following Jira installations:

    JIRA v7.10.1 Server
    JIRA v8.0.1 Server
    Jira V505a6959 Cloud

    Licenses

    The TestBench CS offers an overview of current purchased licenses.
    The details of the licenses are displayed on the licenses dialog which can be accessed only by the administrator.
    The Licenses option is shown in the tenant menu.


    Licenses option on tenant menu



    The licenses dialog displays available licenses.
    A section 'Modules' shows licenses for modules which can be purchased seperatly.



    Licenses Details Dialog

    System Requirements

    The TestBench CS user interface is tested for several browsers but it is impossible to optimize the code of the interface for all existing browsers. Therefore, the system can only be accessed by a limited number of browsers.

    Fully supported browsers:

    Google Chrome 66.0 and newer versions (recommended browser)
    Mozilla Firefox 60.0 and newer versions
    Mozilla Firefox ESR 60.0 and newer versions
    Opera 54.0 and newer versions

    Limited supported browsers:

    Google Chrome 55.0 and newer versions lower than 66.0
    Mozilla Firefox 52.0 and newer versions lower than 60.0
    Mozilla Firefox ESR 52.0
    Apple Safari 11.0.3
    Opera 51.0
    Microsoft Edge 15.15063 and newer versions
    Microsoft Internet Explorer 11.0

    When a limited supported browser is used the system alerts this and grants access only by clicking an ENTER AT YOU OWN RISK confirmation button. For this browsers the vendor does not guarantee a bug free user experience. It is strongly recommended to use one of the fully supported browsers.

    For all other browsers the access to the system is blocked and there is no guarantee that this browser will run the system in any way.

    To use the system it is required that the local storage of the browser must be accessible and that JavaScript must be switched on.
    When one of these requirements are not fulfilled the system denies the access.

    The WSS-Protocol needs to be enabled within the TestBench CS users network.

    For any questions, please send an email to support@testbench.com.