Skip to main content

Executing automated tests

You can execute all test sets using the Execute > Execute Automated option from the Overview tab of a test set. This option invokes Orchestrator for test cases created in Test Manager.

You can also execute automated tests if the test set includes test cases linked from Studio. For more information, check Executing Tests.

Prerequisites​

To execute automated tests while the Allow legacy execution context setting is disabled, you need the following permissions in Orchestrator:

  • At the tenant level: Folders.View and Users.View.
  • At the folder level, where the test automation package was published: TestSets.Create and TestSetExecutions.Create.

Conditions​

important

To use the Execute Automated option, you need at least one automated test case in a test set.

  • Test execution for test sets linked from Orchestrator includes only the test cases published to the Orchestrator Tenant Process Feed in the latest package version. To execute new test cases, publish a new package version in Orchestrator, and link the test set to Test Manager.
    important

    For any tenant, test management capabilities have been moved to Test Manager. As a result, test schedules are no longer available in Orchestrator, and you cannot execute test cases and test sets directly from Orchestrator anymore. However, you can continue executing test cases and test sets through Orchestrator APIs for Test automation, or using the native Test Manager - CI/CD integration.

  • The following information is applicable only if you are using the legacy execution context in Test Manager: If your Test Manager test set includes test cases from multiple Studio projects, then the project that has the highest number of test cases is considered for execution. As a result, only the test cases from this project will be executed. Additionally, Test Manager will display a warning stating that multiple packages have been selected, and only some of them will be executed. You can execute test sets including test cases from multiple studio projects only if you link Studio automations to Test Manager test cases using only the Configure Automation functionality.
  • If your test set includes duplicate test cases in multiple Studio projects, the sorting is done by the project name in descending order. Only the test cases from the project that is placed first in the list will be executed.

Execute Automated Test Sets​

  1. Open your project in Test Manager and then navigate to Test Sets.
  2. Select a test set and click the vertical ellipsis.
  3. Select Execute Automated.

For more information on test results, see Analyzing Test Results.

note
  • Test cases that cannot be executed are skipped during execution through Orchestrator.
  • Test cases that are not executed generate test case logs with no result.

Alternatively, you can open your test set, click Execute and select Automated from the dropdown.

Configuring the default test execution timeout​

To change the default 12-hour execution timeout, follow these steps:

  1. Locate the configmap file in your Orchestrator Automation Suite installation folder.
  2. Open the file, and add the TestAutomation.TestSetExecutionTimeoutMinutes setting. 
    <add key="TestAutomation.TestSetExecutionTimeoutMinutes" value="40" />
  3. The value of the TestAutomation.TestSetExecutionTimeoutMinutes setting is the required timeout duration. The value, expressed in minutes, represents the duration after which your test executions are marked as Cancelled.