Retry Flaky Tests

We now support two new parameters in the build API.

retryTestsOnFailure : This is a boolean and it maps to the -retry-tests-on-failure param in XCode. This will make sure we retry the test-case only in case the test result is a failure and it will continue to do so, until either it is a success or the number of attempts is equal to the value of the param mentioned below.

testIterations : This is an integer, and should be > 0 and <=100. This maps to the -test-iterations param in XCode. This specifies the number of times a specific test case must be run in case the test case is not successful.

The status of a test-case(Seen in the API or Dashboard) which has been run multiple times will be the status of the last attempt/iteration.

You can use these two parameters to retry failed tests. When flaky tests are run again, many tests which were failures end up passing. This increases the pass percentage in the session and in the build.

Sample API request

This is an example of our BUILD API which also uses these retry params(testIterations=5 and retryTestsOnFailure=true).

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/xcuitest/v2/build" \
-d '{"devices": ["iPhone 13-15"], "app": "bs://9eff01e4a24772a3a40cd8ed1aa5cb7744d1fea4", "testSuite": "bs://c744336031c922f7a80d410b634f7b657dec64b3", "retryTestsOnFailure": true, "testIterations": 5 }' \
-H "Content-Type: application/json"