Retry Flaky Tests

We now support three new parameters in the build API.

retryTestsOnFailure : This is a boolean and it maps to the -retry-tests-on-failure param in XCode. This will make sure we retry the test-case only in case the test result is a failure and it will continue to do so, until either it is a success or the number of attempts is equal to the value of the param mentioned below.

testIterations : This is an integer, and should be > 0 and <=100. This maps to the -test-iterations param in XCode. This specifies the number of times a specific test case must be run in case the test case is not successful.

testRelaunchEnabled : This is a boolean. When this flag is used, the test runner process is spawned in a different process. You should use this flag along with retryTestsOnFailure and testIterations when you receive any of the following errors:

• Failed to establish communication with the test runner
• Test crashed with signal trap/abrt/kill

The status of a test-case (Seen in the API or Dashboard) which has been run multiple times will be the status of the last attempt/iteration.

You can use these parameters to retry failed tests. When flaky tests are run again, many tests which were failures end up passing. This increases the pass percentage in the session and in the build.

Sample API request

This is an example of our BUILD API which also uses these retry params(testIterations=5 and retryTestsOnFailure=true).

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/xcuitest/v2/build" \
-d '{"devices": ["iPhone 13-15"], "app": "bs://9eff01e4a24772a3a40cd8ed1aa5cb7744d1fea4", "testSuite": "bs://c744336031c922f7a80d410b634f7b657dec64b3", "retryTestsOnFailure": true, "testIterations": 5 }' \
-H "Content-Type: application/json"

Related topics

• Manage your uploaded app or test suite using our upload app REST API and upload test suite REST API.

• Use the get app API endpoint and list test suites API endpoint to list your recently uploaded apps and test suites.

• Delete your uploaded app or test suite using the delete app API endpoint and delete test suite API endpoint.