For AI agents and LLMs: a machine-readable index is available at llms.txt. A plain-Markdown version of any documentation page is available by appending .md to its URL.
Skip to main content

Test Run - Creation and Management

This guide outlines the steps required to create, configure, and manage test runs within TestMu AI's Test Manager. It provides developers and testers with a clear understanding of the process, enabling efficient test execution and organization.

1. Creating a Test Run​

Initiate a New Test Run​

  • Navigate to the Test Manager and click on Create Test Run.
  • Enter a Test Run Name and an optional Description to define the purpose of the test run.
tip

Use descriptive names to easily identify test runs later.

Real

Add Tags​

  • Assign relevant Tags to categorize the test run.
  • Click Create Test Run to proceed.
Real

Creating the test run takes you through three steps: Enter Details, Add Test Cases, and Set Configurations and Assignee. The steps are shown at the top of the page, so you can see which one you are on.

2. Adding and Configuring Test Cases​

Add Test Cases​

In the Add Test Cases step, choose the test cases to include in the test run.

  • Use the Folders tree to work folder by folder. Each folder shows how many of its test cases are selected, and selecting a folder's checkbox selects every test case in it.
  • Use search and the Status, Tags, and More filters to narrow the list. Most filters can also exclude values: switch one to Not Equals to leave out test cases with the selected values, while test cases with no value for that field stay in the list. See Include or Exclude Values.
  • Select test cases one by one, or use the checkbox in the list header to select every test case the list shows. With filters applied, select all picks exactly the test cases the filters show.
  • The number of selected test cases is shown at the bottom of the page. Use Clear All to start over.
  • Click Add Test Cases to add the selection and move to the next step.

The same picker opens when you add or remove test cases later from Manage Test Cases.

Add Test Cases step with the folder tree, filters, and every test case selected Status filter set to Not Equals in Manage Test Cases
tip

Test cases shared into your project from other projects can be added to a test run too. Select them from Shared Incoming. See Share Test Cases Across Projects.

Set Configurations and Assignee​

In the Set Configurations and Assignee step, every added test case is listed along with what it still needs. A test case is flagged when it has no configuration, or when some of its test instances have no assignee. Below each test case, a summary shows how many configurations and test instances it has, and how many of those test instances are assigned. Expand a test case to see its configurations.

For each test case, you can:

  • Set Config: choose the configurations to run it on. Each configuration creates one test instance of the test case. Once a test case has configurations, use the same action to update them.
  • Set Assignee: assign its test instances to a team member.
  • Delete: remove the test case from the test run.

To change many test cases at once, select them, or select all of them from the list header, and use the same actions in the header.

The summary bar at the bottom of the page shows the number of test cases added, the number of test instances, how many test cases are missing a configuration, how many test instances are missing an assignee, and the configurations in use.

To add or remove test cases without leaving this step, click Manage Test Cases.

Set Configurations and Assignee step with every test case selected and configurations and assignees still missing

Add Configurations​

Set Config opens the Add Configuration panel.

  • Search configurations by name, or filter them by OS, browser, and resolution.
  • Select the configurations to apply. You can select several, or all of them.
  • Click Create a Configuration to add one that does not exist yet, or Manage configurations to open your configuration list.
  • Click Apply.

Note: Configurations allow you to define environment settings, such as browser and device combinations.

warning

Applying configurations to several test cases at once replaces the configurations those test cases already have.

Add Configuration panel with every configuration selected

To run a parameterized test case once for each dataset row, set a dataset on its configuration. See Run Parameterized Manual Test Cases.

Large Test Runs Are Processed in the Background​

When you add a large number of test cases, or apply configurations or assignees to a large number of them, the change is applied in the background. The page stays responsive while you set up large test runs, instead of hanging or failing.

  • While a change is being applied, a progress indicator is shown at the top of the step. You can wait on the page, or continue working elsewhere and come back later.
  • Save Test Run is unavailable until the change finishes.
  • When the change finishes, the list and the summary bar are updated. For example, applying 40 configurations to 91 test cases creates 3,640 test instances.
Configurations being applied in the background, with Save Test Run unavailable Set Configurations and Assignee step after 40 configurations were applied to 91 test cases

3. Saving and Managing Test Runs​

Finalize the Test Run​

  • Click Save Test Run to finalize the setup.
  • View the created test run and its associated test instances in the dashboard.

Test instances are organized using a Folders sidebar on the left, reflecting the folder structure of the test cases added to the test run. Only folders whose test cases are included in the test run are displayed, and each folder shows the count of test instances it contains (e.g., 3/3, 4/4).

  • Select a folder to filter and view only its test instances. The status summary (Passed, Failed, Not Started, Skipped) is displayed in the top-right corner of the listing for the selected folder.
  • Collapse or expand folders for easier navigation.
  • Click the three-dot menu on the Folders header to toggle Include Child Folders. When enabled (default), selecting a parent folder also displays test instances from its child folders.
Real

Bulk Update Options​

  • To update multiple test instances:
  • Select multiple test instances.
  • Use the Select Assignee and Select Status options to apply changes in bulk.
Real

Setting statuses here counts as setting them by hand, so those test instances are no longer marked as derived until their steps change again. See Setting the Test Instance Status Yourself.

Update Test Step Status​

  • Within any test instance, individually update the status of each test step.
  • Add remarks or actual outcomes for manual test steps.
  • Move a step back to Not Started if it has to be executed again.

On a manual test instance, marking a step also updates the status of the test instance itself. See How the Test Instance Status Is Derived.

Real

How the Test Instance Status Is Derived​

On a manual test instance, changing the status of a step recalculates the status of the instance from all of its steps, not only the one you changed. You record the verdict once, on the step, and the instance follows.

The instance status is written only when the recalculated value differs from the one already stored, so nothing else about the instance changes.

The rules below are checked in order, and the first one that matches decides the status.

OrderIfThe test instance becomes
1Any step is FailedFailed
2Any step has not been executed yet — Not Started, or not marked at allNot Started
3Every step is executed and at least one is on a custom statusThat custom status, from the lowest-numbered step holding one
4Every step is executed and at least one is PassedPassed
5Every step is executed and none of the above appliesSkipped

Examples​

Blocked and Deferred below are examples of custom statuses your project may have added to the Test Run Status field.

Step statuses, in orderTest instance statusWhy
Passed, Passed, PassedPassedEvery step was executed and passed.
Passed, Failed, PassedFailedOne failed step is enough, wherever it is.
Failed, Failed, SkippedFailedMore than one failure makes no difference.
Blocked, Failed, PassedFailedA failure outranks a custom status.
Passed, Not Started, Not StartedNot StartedSteps are still unexecuted.
Passed, Blocked, Not StartedNot StartedAn unexecuted step outranks the custom status.
Not Started, Not Started, Not StartedNot StartedNothing has been executed yet.
Not marked, Passed, SkippedNot StartedA step with no status counts as unexecuted.
Passed, Blocked, SkippedBlockedEvery step is executed, so the custom status decides.
Blocked, Passed, SkippedBlockedA custom status is never resolved into Passed.
Blocked, Deferred, PassedBlockedWith two custom statuses, the lowest-numbered step wins.
Skipped, Skipped, BlockedBlockedA custom status outranks Skipped.
Passed, Skipped, PassedPassedA skipped step does not block a pass.
Skipped, Skipped, SkippedSkippedEverything was executed, and nothing passed or failed.

Custom Statuses in a Derived Status​

Custom statuses are the extra values your project adds to the Test Run Status system field. See System and Custom Fields.

  • A custom status is an end state. It is never resolved into Passed by a later rule, and the execution end time is stamped on the instance when it is derived.
  • Renaming a custom status applies the new name on the instance and on its steps, and the instance stays marked as derived.
  • Deleting a custom status, or unlinking a project from it, resets every instance holding it to Not Started and leaves the steps that carried it unmarked.

Setting the Test Instance Status Yourself​

  • You can still set the test instance status by hand at any time, and it is kept exactly as you set it.
  • A status you set by hand is no longer marked as derived. The next real step status change derives over it.
  • Re-selecting the status a step already has is not a change, so it triggers no recalculation and leaves a hand-set instance status alone.
  • Setting statuses in bulk from the test instance list counts as setting them by hand. See Bulk Update Options.
  • Updating several steps in one bulk action produces a single recalculation, not one per step.
  • Moving a step back to Not Started pulls the instance out of an end state such as Passed, Failed or a custom status, and clears its execution end time.

Where Derivation Does Not Apply​

  • Automation and KaneAI test instances, whose status is reported by the runner.
  • Test instances in an archived test run.
  • Test instances that have no steps. Their status stays manually settable.
  • Test instances already on Timed Out, Aborted, Lambda Error or Muted. These are never overwritten.
note

A derived status change is recorded on the test instance audit log against the person who changed the step, naming the step that drove it. See Test Instance Audit Logs. Syncing an instance to a newer test case version resets it and stops it being marked as derived, until its next step status change. See Sync Test Instances.

A derived status rolls up to the test run in the same way a hand-set one does, so the test run status and its status counts stay in step with what was executed.

Turn Status Derivation On or Off​

Deriving the test instance status from step results is enabled by default for every organization and applies to all of its projects.

This behaviour is configured at the Organization level within the Org Product Preferences section of Organization settings, under Test Manager > Manual Test Status, here. Use the Set test instance status from step results toggle, then save your changes.

Manual Test Status setting under Test Manager in Org Product Preferences
  • Turn it off and no test instance status is derived any more. Statuses that were already derived are left as they are.
  • Turn it on and it applies from each instance's next step status change. Nothing is recalculated or rewritten for executions that already happened.
note

Only Admins of the Organization can change this setting.

Read the Test Case Details During Execution​

A manual test instance opens on three tabs: Test steps, Test case details, and Issues.

The Test case details tab shows the content of the test case you are executing, so you do not have to leave the run to read it:

  • Description, Pre-conditions, and Attachments, which you can preview or download.
  • Status, Type, Priority, Automation Status, Tags, and any Custom Fields.

Fields that have no value are left out. The tab is read only: you cannot edit the test case or add attachments from here. To change any of it, open the test case from its ID in the instance header and edit it on the Test Case Details page.

Test case details tab on a manual test instance

The tab shows the test case as of the version this instance is linked to, the version badge next to the test case ID, which is not always the latest version of the test case. When you update the instance to a newer version, the tab reflects the new version along with the steps. See Sync Test Instances.

Applies to Manual Test Runs Only

The Test case details tab is available on manual test instances.

4. Filtering Test Instances​

A filter bar above the test instance list lets you narrow down a test run by:

  • Test instance attributes: Status, Assignee, Platform, OS, Browser, Resolution, Device.
  • Test case attributes: Tags, Priority, Test Case Status.
  • Test Case Custom Fields: available under More Filters.

Filters work on both Manual and Automation test runs, apply at the All Test Instances view as well as at any folder, and persist as you switch between folders. The active filter state is also reflected in the page URL, so you can bookmark or share a filtered view.

Filter bar in the Test Run Detail view
Supported Custom Field types

Custom Fields can be filtered only for types: Single Select Dropdown, Multi Select Dropdown, User, Boolean, and Date. Of these, Single Select Dropdown, Multi Select Dropdown, and User can also be excluded with Not Equals.

Exclude Values​

Each filter that lists values can either include or exclude the values you select. Switch a filter to Not Equals next to its search box to hide the test instances that have any of the selected values, for example every instance on Windows, or every instance assigned to one tester.

Not Equals keeps test instances that have no value for that field. Excluding an assignee keeps the unassigned instances, and excluding an OS or browser keeps the instances with no configuration. To hide the unassigned instances as well, also select Unassigned in the Assignee filter.

Filters in different fields combine, so an instance has to match all of them. When you select all instances in a filtered view, bulk updates apply only to the instances the filters show. For how exclusion works on each kind of field, see Include or Exclude Values.

5. Enhancing Test Runs with Test Evidences​

Adding Remarks and Attachments​

  • Add Remarks or actual outcomes to enhance test instance execution details.
  • You can add remarks and attachments at both the individual test step level and the overall test instance level.
Real

The Remark field supports rich text formatting, including:

  • Inline image embedding: paste images directly into the field.
  • Text styling such as bold, italic, bullet lists, and other standard rich text options.
note

Remarks have a maximum limit of 5000 characters.

  • Attach supporting files or screenshots to the remarks for better context.
Real

6. Execute Test Runs on TestMu AI Cloud​

Execute your manual test instances directly on TestMu AI Cloud, no local setup or environment configuration required. This allows your team to run manual tests on real browsers and devices hosted on the cloud, making it one of the most efficient ways to validate your test cases.

To execute a test run:

  1. Click the Play icon on any test instance to launch execution on TestMu AI Cloud.
Real
  1. During execution, update the status of individual test cases and test steps in real time as you verify each one.
Real
Why execute on the Cloud?

Running test instances on TestMu AI Cloud gives you access to a wide range of real browsers, devices, and OS combinations, without maintaining local infrastructure. It ensures consistent, reliable test execution across environments.

Track bugs during execution

Bugs raised or linked while executing an instance are surfaced under that Test Case instance. To link Jira, Azure DevOps, or Linear tickets at the instance and step level, see Track Bugs and Issues in Test Runs.

7. Test Run Options​

You can manage your test runs using the options available in the three-dot menu on each test run. The following actions are available:

ActionDescription
EditModify the test run by adding or removing test cases from Manage Test Cases, or by changing configurations and assignees. Large changes are processed in the background. KaneAI test runs can only be edited if they have not been executed.
DuplicateCreate a copy of the test run for re-execution or variation testing.
ArchiveMove completed test runs to the archive to keep your workspace clean and organized. Test instances in an archived test run do not have their status derived from step results.
DeletePermanently remove a test run that is no longer needed.
note

Archiving a test run is not the same as archiving a test case. An archived test case stays in any run it was already part of and keeps its executions as history, but it is not offered when you create a new test run and is not carried across when you duplicate one. See Archive and Restore Test Cases.

Real Editing an existing test run
Act on many test runs at once

Select several test runs in the list to move them to another folder, copy them into fresh runs for the next cycle, or delete them in one action. You can also drag selected runs onto a folder to move or copy them. A moved run keeps its ID and execution history, and a copied run starts with every test instance at Not Started. See Bulk Move, Copy, and Delete Test Runs.

Terminal First Testing With Kane CLI

Natural language browser & mobile app tests right from terminal.

×
Schedule Your Personal Demo
Kane CLI terminal

Help and Support

Related Articles