Ask the Community
Retry your flaky XCUI tests
Increase your test pass percentage by retrying failing XCUI tests.
Flaky tests fail intermittently because of transient issues rather than real bugs. To increase your test pass percentage, you can automatically retry failed XCUI test cases using the following parameters in the builds REST API:
| Parameter | Type | Description |
|---|---|---|
retryTestsOnFailure |
Boolean | Maps to the -retry-tests-on-failure parameter in Xcode. Retries a test case only when the test result is a failure, and continues to retry until the test passes or the number of attempts equals testIterations. |
testIterations |
Integer | Maps to the -test-iterations parameter in Xcode. Specifies the number of times a specific test case runs when it is not successful. The value must be greater than 0 and less than or equal to 100. |
testRelaunchEnabled |
Boolean | Spawns the test runner in a separate process. Use this flag along with retryTestsOnFailure and testIterations when you receive either of the following errors:Failed to establish communication with the test runnerTest crashed with signal trap/abrt/kill
|
The testIterations and retryTestsOnFailure parameters are not supported on iOS versions earlier than 15.
The status of a test case that runs multiple times, shown in the API or on the App Automate dashboard, is the status of the last attempt.
You can use these parameters to retry failed tests. When flaky tests run again, many tests that previously failed end up passing. This increases the pass percentage in the session and in the build.
Retrying failed tests is not supported when singleRunnerInvocation is set to true. Because all test cases run together in a single process, BrowserStack cannot isolate and rerun an individual failed test, so the retryTestsOnFailure and testIterations parameters have no effect. To use these parameters, keep singleRunnerInvocation set to false, which is the default. For more information, see Single runner invocation.
The following cURL request uses the builds REST API with the retry parameters testIterations=5 and retryTestsOnFailure=true:
Related topics
-
Manage your uploaded app or test suite using our upload app REST API and upload test suite REST API.
-
Use the get app API endpoint and list test suites API endpoint to list your recently uploaded apps and test suites.
-
Delete your uploaded app or test suite using the delete app API endpoint and delete test suite API endpoint.
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
Thank you for your valuable feedback!