Skip to main content
No Result Found

Test orchestration on BrowserStack Automate

Optimize your Automate tests by efficiently managing their execution, reporting, and rerunning.

Test Orchestration is designed to streamline your team’s testing processes. It enhances test execution by automating tasks such as rerunning failed tests, stopping test suites after critical failures, and prioritizing failed tests. These features reduce manual intervention, optimize resource usage, and improve the reliability of test outcomes, allowing teams to focus on resolving defects faster.

With Test Orchestration, you can accelerate feedback loops, quickly identify and address critical issues, and ensure your development and QA teams maintain high product quality throughout the testing cycle.

The current frameworks supported by BrowserStack Test Orchestration are TestNG and JUnit-5 for Java.

Supported strategies

BrowserStack Automate currently supports the following Test Orchestration strategies:

Test Orchestration is BrowserStack can be triggered using the following capability:

Capabilities Description Expected values
testOrchestrationOptions Enable Test orchestration. retryTestsOnFailure - Automatically rerun failed tests.
abortBuildOnFailure - Stops the execution of a test suite when a critical failure occurs.
runPreviouslyFailedFirst - Prioritize running the previously failed tests in the suite first.

Key Features

Auto Reruns

Automatically rerun failed tests to identify flaky issues, reduce manual intervention, and improve test reliability. By automatically rerunning failed tests, you can address intermittent issues without manually triggering additional test cycles, ensuring more reliable results.

To enable Auto Reruns, configure the feature in your browserstack.yml file. You can control the number of reruns by setting a limit on how many times a test should be retried.

browserstack.yml
Copy icon Copy

Fail Fast

Fail Fast stops the execution of a test suite when a critical failure occurs. This feature helps teams save resources and prioritize fixing high-priority issues by preventing further execution of tests that are likely to fail due to existing critical problems.

You can configure Fail Fast by setting thresholds for the number of allowable failures. Once this threshold is reached, the test suite will automatically abort, allowing the team to focus on resolving the most urgent issues.

browserstack.yml
Copy icon Copy

Prioritizing Failures

Prioritizing failures involves running previously failed tests first in subsequent test cycles. This strategy is useful when faster feedback is crucial, as it helps teams quickly identify if known issues persist or if new issues have emerged.

By prioritizing fixing failures from the last test run, you can expedite feedback and address high-impact bugs earlier in the development process.

browserstack.yml
Copy icon Copy

Benefits

  • Faster Feedback Loops Running failed tests first helps teams get immediate insights into critical issues, reducing the time taken to detect and fix problems.
  • Increased Test Efficiency By focusing on high-priority failures, teams can avoid re-running all tests unnecessarily, saving time and resources.
  • Early Bug Detection Prioritizing failures ensures that critical issues are addressed sooner, minimizing the risk of issues progressing further into the development cycle.
  • Improved Development Focus Identifying recurring or new failures quickly allows developers to concentrate on resolving issues that have a higher likelihood of impacting the end-user experience.

Prioritizing failures is currently supported only for TestNG, and in master-slave setups, it is supported with Jenkins CI/CD.

The following is a sample master-slave script for a Jenkinsfile in a Jenkins Pipeline:

Jenkinsfile
Copy icon Copy

BrowserStack will be adding more strategies to the above list. If you want any specific Test Orchestration feature, contact support.

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked





Thank you for your valuable feedback

Is this page helping you?

Yes
No

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked





Thank you for your valuable feedback!

Talk to an Expert
Download Copy Check Circle