Skip to main content
Experience faster, smarter testing with BrowserStack AI Agents. See what your workflow’s been missing. Explore now!
No Result Found
Connect & Get help from fellow developers on our Discord community. Ask the CommunityAsk the Community

Test orchestration & selection on BrowserStack Automate

Optimize your Automate tests by efficiently managing their execution, reporting, and rerunning.

As test suites grow and release cycles speed up, it’s important to optimise what gets tested and how. Test Selection and Orchestration help optimise your build and achieve the benefits of:

  • Faster feedback and faster product releases through targeted and prioritised runs
  • Higher QA productivity via reduced debugging of flaky and non-relevant tests.
  • Reduced infrastructure usage & cost by skipping less relevant tests. The BrowserStack SDK allows you to configure strategies such as auto retries, fail fast, rerun failures only/first, etc., to optimise your test builds.

Benefits of test orchestration

  • Faster feedback loops: Running failed tests first helps teams get immediate insights into critical issues, reducing the time taken to detect and fix problems.
  • Increased test efficiency: By focusing on high-priority failures, teams can avoid re-running all tests unnecessarily, saving time and resources.
  • Early bug detection: Prioritizing failures ensures that critical issues are addressed sooner, minimizing the risk of issues progressing further into the development cycle.
  • Improved development focus: Identifying recurring or new failures quickly allows developers to concentrate on resolving issues that have a higher likelihood of impacting the end-user experience.

The following are supported testing frameworks:

  • TestNG and JUnit-5 for Java
  • Pytest for Python
  • Jest for Node
  • Nunit for C#

Using the BrowserStack SDK to integrate with Automate offers significant benefits in stability, performance, and ease of management. Learn how to integrate your tests with BrowserStack SDK.


From the Automate dashboard, you can expand the Test Orchestration menu to see the time saved by each strategy and begin configuration. The Test Orchestration menu expanded on the BrowserStack Automation dashboard, showing options to configure strategies and view time saved.

Supported strategies

BrowserStack Automate supports the following strategies, which you can enable using the testOrchestrationOptions capability in your browserstack.yml file. The strategies are mutually exclusive of each other.

Capability Description
retryTestsOnFailure Automatically rerun failed tests.
abortBuildOnFailure Stops the execution of a test suite when a critical failure occurs.
rerunPreviouslyFailed Run only the tests that failed in the previous build.
skipFlakyandFailed Skips tests based on criteria like previous failures or flakiness.
runPreviouslyFailedFirst Prioritize running the previously failed tests in the suite first.


Clicking Configure opens a modal where you can toggle your preferred strategies and copy the generated browserstack.yml code. Configuration modal for test orchestration, with toggles for strategies like Auto Reruns and Fail Fast, and a ready-to-copy browserstack.yml code snippet.

Auto reruns

Automatically reruns failed tests to identify and manage flaky tests. This strategy helps you confirm whether a failure is persistent or intermittent, improving the reliability of your test results without manual intervention.

To enable auto reruns, configure the feature in your browserstack.yml file. You can control the number of reruns by setting a limit on how many times a test should be retried.

browserstack.yml
Copy icon Copy

Fail fast

Fail fast stops the execution of a test suite when a critical failure occurs. This feature helps teams save resources and prioritize fixing high-priority issues by preventing further execution of tests that are likely to fail due to existing critical problems.

You can configure Fail Fast by setting thresholds for the number of allowable failures. Once this threshold is reached, the test suite will automatically abort, allowing the team to focus on resolving the most urgent issues.

browserstack.yml
Copy icon Copy

Run failures only

Reruns only the tests that failed in the previous build. This provides a rapid feedback loop for developers and QA engineers who need to quickly verify a bug fix without executing the entire test suite.

Although this selection strategy is not yet reflected on the dashboard, you can enable it through the capabilities configuration. This will execute all tests that match the selection criteria, allowing you to benefit from faster feedback.

You can enable this strategy in your browserstack.yml file.

browserstack.yml
Copy icon Copy

Skip flaky and failing

Runs a test suite while temporarily ignoring tests that are known to be failing or flaky. This is useful when you want to assess the overall health of an application without being blocked by known issues, such as tests that are failing intermittently or are awaiting a fix.

Although this selection strategy is not yet reflected on the dashboard, you can enable it through the capabilities configuration. This will execute all tests that match the selection criteria, allowing you to benefit from faster feedback.

browserstack.yml
Copy icon Copy

Prioritizing failures

Prioritizing failures involves running previously failed tests first in subsequent test cycles. This strategy is useful when faster feedback is crucial, as it helps teams quickly identify if known issues persist or if new issues have emerged.

By prioritizing fixing failures from the last test run, you can expedite feedback and address high-impact bugs earlier in the development process.

browserstack.yml
Copy icon Copy


Once configured, the main build screen displays which orchestration strategies, like Auto Rerun and Prioritizing failures, were active for that specific run.

Test orchestration configuration modal showing active strategies and saved results

Prioritizing failures is currently supported only for TestNG, and in master-slave setups, it is supported with Jenkins CI/CD.

Example: Jenkins pipeline integration

To see how Test Orchestration can be applied in a CI/CD environment, you can use the following Jenkinsfile script. This example configures a parallel pipeline that distributes the test suite across multiple nodes, passing the required BrowserStack environment variables to each job for seamless integration.

Jenkins
Copy icon Copy
Jenkins
Copy icon Copy
Jenkins
Copy icon Copy
Jenkins
Copy icon Copy

The list of supported strategies is actively being expanded. If you have a requirement for a specific Test Orchestration feature that is not listed, we encourage you to submit a feature request by contacting BrowserStack support.

Smart Test Selection

Smart Test Selection uses AI to choose and run a small, prioritized subset of your tests based on recent code changes. This gives you faster feedback on builds and improves test efficiency. Learn how to use BrowserStack’s Smart Test Selection

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked





Thank you for your valuable feedback

Is this page helping you?

Yes
No

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked





Thank you for your valuable feedback!

Talk to an Expert
Download Copy Check Circle