Test orchestration & selection on BrowserStack App Automate

Mobile app test optimization on BrowserStack App Automate. Orchestrate test runs, select the right tests, analyze reports, and rerun failures efficiently.

As test suites scale, running every test on every build can slow down release cycles and create CI bottlenecks. To maintain fast and reliable delivery, it becomes essential to optimize what tests run and how without compromising confidence.

Test Selection and Orchestration determine which tests run and define how those tests are executed.

From the App Automate dashboard, you can click Try now in the Build Optimization widget, which opens a modal showing time saved by each strategy. Toggle your preferred strategies and copy the generated code in your browserstack.yml file to enable them

Supported strategies

BrowserStack App Automate supports the following strategies, which you can enable using the testOrchestrationOptions capability in your browserstack.yml file.

Strategy	Description
Auto rerun	Immediately retries failed tests for a configurable number of attempts.
Fail fast	Stops the execution of a test suite once a failure threshold is reached.
Run failures only	Runs only the tests that failed in the previous build.
Prioritizing failures	Prioritizes running the previously failed tests in the suite first.
Skip flaky and failing	Skips tests based on criteria like previous failures or flakiness.
Smart Test Selection	Runs likely-to-fail tests based on code changes.

Learn how to combine different orchestration strategies here

Auto reruns

Automatically retry tests when they fail to identify and manage flaky tests. This strategy helps you confirm whether a failure is persistent or intermittent, improving the reliability of your test results without manual intervention.

To enable auto reruns, configure the feature in your browserstack.yml file. You can control the number of reruns by setting a limit on how many times a test should be retried.

Fail fast

Fail fast stops the execution of a test suite when a critical failure occurs. This feature helps teams save resources and prioritize fixing high-priority issues by preventing further execution of tests that are likely to fail due to existing critical problems.

You can configure Fail Fast by setting thresholds for the number of allowable failures. Once this threshold is reached, the test suite will automatically abort, allowing the team to focus on resolving the most urgent issues.

Run failures only

Reruns only the tests that failed in the previous build. This provides a rapid feedback loop for developers and QA engineers who need to quickly verify a bug fix without executing the entire test suite.

Skip flaky and failing

Runs a test suite while temporarily ignoring tests that are known to be failing or flaky. This is useful when you want to assess the overall health of an application without being blocked by known issues, such as tests that are failing intermittently or are awaiting a fix.

Prioritizing failures

Prioritizing failures involves running previously failed tests first to get faster feedback and identify bugs early. Prioritizing failures helps you quickly determine if issues persist or if new defects emerge. This approach ensures you address high-impact bugs early in the development process.

Smart Test Selection

Smart Test Selection uses AI to select and run a subset of likely-to-fail tests based on recent code changes. This gives you faster feedback on builds and improves test efficiency. Learn how to use BrowserStack’s Smart Test Selection.

View Results

Once configured, the main build screen displays active orchestration strategies and time saved by these strategies for each run.

Supported Orchestration Strategy Combinations

You can combine different orchestration strategies to optimize your testing workflow for speed, stability, or efficiency. While many valid combinations are possible to suit specific requirements, the following table highlights the most effective ones for common workflows

Combination	Use Case
Smart Test Selection (relevantFirst) + Fail fast	Best for getting faster feedback on tests relevant to code changes while ensuring the full suite runs unless a critical failure is detected early.
Run failures only + Fail fast	Use for rapid debugging to verify fixes for previous failures and stop the build if those critical tests still fail.
Auto rerun + Fail fast	Automatically retries failed tests and aborts the build if failures persist beyond the configured threshold of allowable failures.

The list of supported strategies is actively being expanded. If you have a requirement for a specific Test Orchestration feature that is not listed, we encourage you to submit a feature request by contacting BrowserStack support.

Benefits of Test Orchestration & Selection

• Faster feedback loops Prioritizing relevant or previously failed tests surfaces critical issues earlier, reducing the time to detect and fix problems and preventing issues from progressing further into the development cycle

• Increased test efficiency Running only relevant tests or aborting build on failures avoids unnecessary test execution, saves time and reduces CI resource usage

• Improved development focus Identifying recurring or new failures quickly allows developers to focus on issues that are more likely to affect end-user experience

Example: Jenkins pipeline integration

You can easily scale your testing by applying Test Orchestration in a CI/CD environment. The following Jenkinsfile script example configures a parallel pipeline that distributes the test suite across multiple nodes. By passing specific BrowserStack environment variables to each job, the orchestration server handles the heavy lifting of splitting tests and syncing results across different nodes automatically Configure these variables when using any orchestration strategy to ensure the orchestration server efficiently manages test distribution across your infrastructure