Speed up Espresso tests using test sharding

Learn how to speed up your Espresso mobile automation tests on BrowserStack App Automate using test sharding. Run tests on real device cloud.

Parallel testing enables you to reduce the overall test execution times, resulting in improved build times and faster releases. Test sharding enables you to test at scale on BrowserStack.

Introduction

By default, all test cases in your Espresso test suite are executed sequentially. Instead of running your test suite sequentially, you can split test cases into different groups called shards. Each shard can then be executed in parallel on one or more devices. This is especially helpful when your test suite has a large number of test cases.

For example, assume your test suite has 500 test cases and each test case takes about a minute to execute. If the test suite runs sequentially, the overall execution time is close to 500 minutes. However, if you split your test suite into 10 shards, each shard consists of 50 test cases. If each shard is executed in parallel, the total test execution time comes down to 50 minutes.

To enable test sharding, configure the shards parameter in the REST API request to start Espresso test execution.

REST API endpoint:

POST /app-automate/espresso/v2/build

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2}, "devices": ["Google Pixel 7-13.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://j3c874f21852ba57957a3fdc33f47514288c4ba4"}' \
-H "Content-Type: application/json"

In this example, the test suite is split into 2 shards, and each of those shards executes in parallel on Google Pixel 7-13.0. This blocks two parallel test licenses in your App Automate subscription plan.

Sharding strategy

There are multiple strategies to control how you want to split the test suite into different shards.

1. Use auto strategy

This is the simplest way to use test sharding. The test suite is automatically split into the desired number of shards. Each shard has approximately the same number of test cases. BrowserStack relies on the AndroidJUnitRunner shard feature to implement this behavior. All the shards execute in parallel on all the devices specified in the devices parameter.

REST API endpoint:

POST /app-automate/espresso/v2/build

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 7-13.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4"}' \
-H "Content-Type: application/json"

In the above example, the test cases are automatically split into 2 groups (i.e. shards). Both the shards execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 7-13.0. This blocks four parallel test licenses in your App Automate subscription plan.

2. Use package strategy

Using this strategy, you can group test cases by their fully qualified Java package names (e.g. com.foo.login) into different shards. All the shards execute in parallel on the devices specified in the devices parameter.

REST API endpoint:

POST /app-automate/espresso/v2/build

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "package", "values": ["com.foo.login", "com.foo.logout"]}, {"name": "Shard 2", "strategy": "package", "values": ["com.foo.dashboard"]}]}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 7-13.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases are split into 2 groups (i.e. shards). The first shard consists of test cases belonging to com.foo.login and com.foo.logout packages. The second shard consists of test cases belonging to the com.foo.dashboard package. Both the shards execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 7-13.0. This blocks four parallel test licenses in your App Automate subscription plan.

3. Use class strategy

Using this strategy, you can group test cases by their fully qualified Java class names (e.g. com.foo.login.user) into different shards. All the shards execute in parallel on the devices specified in the devices parameter.

REST API endpoint:

POST /app-automate/espresso/v2/build

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "class", "values": ["com.foo.login.user", "com.foo.login.admin"]}, {"name": "Shard 2", "strategy": "class", "values": ["com.foo.logout.user"]}]}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 7-13.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases are split into 2 groups (i.e. shards). The first shard consists of test cases belonging to com.foo.login.user and com.foo.login.admin classes. The second shard consists of test cases belonging to the com.foo.logout.user class. Both the shards execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 7-13.0. This blocks four parallel test licenses in your App Automate subscription plan.

4. Use annotation strategy

Using this strategy, you can group test cases by annotations (e.g. com.foo.login.SmokeSuite) into different shards. All the shards execute in parallel on the devices specified in the devices parameter.

REST API endpoint:

POST /app-automate/espresso/v2/build

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "annotation", "values": ["com.foo.login.SmokeSuite"]}, {"name": "Shard 2", "strategy": "annotation", "values": ["com.foo.login.RegressionSuite"]}]}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 7-13.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases are split into 2 groups (i.e. shards). The first shard consists of test cases with the com.foo.login.SmokeSuite annotation. The second shard consists of test cases with the com.foo.login.RegressionSuite annotation. Both the shards execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 7-13.0. This blocks four parallel test licenses in your App Automate subscription plan.

5. Use size strategy

Using this strategy, you can group test cases by specific test size, that is tests annotated with @SmallTest, @MediumTest, or @LargeTest (e.g. small/medium/large) into different shards. All the shards execute in parallel on the devices specified in the devices parameter.

REST API endpoint:

POST /app-automate/espresso/v2/build

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "size", "values": ["small", "medium"]}, {"name": "Shard 2", "strategy": "size", "values": ["large"]}]}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 7-13.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases are split into 2 groups (i.e. shards). The first shard consists of test cases annotated with @SmallTest and @MediumTest. The second shard consists of test cases annotated with @LargeTest. Both the shards execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 7-13.0. This blocks four parallel test licenses in your App Automate subscription plan.

Device selection strategy

If you don’t want each of the shards to execute on all specified devices, you can set the deviceSelection attribute inside the shards parameter to any. This ensures that each shard executes on any randomly selected device from the list of devices specified in the devices parameter. By default, its value is set to all.

REST API endpoint:

POST /app-automate/espresso/v2/build

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "deviceSelection": "any", "mapping": [{"name": "Shard 1", "strategy": "class", "values": ["com.foo.test1", "com.foo.test2"]}, {"name": "Shard 2", "strategy": "class", "values": ["com.foo", "com.foo.login"]}]}, "devices": ["Samsung Galaxy S8-7.0", "Google Pixel 7-13.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases are split into 2 groups (i.e. shards). Both the shards execute in parallel, but each shard executes on a single device randomly selected from the list (i.e. Samsung Galaxy S8-7.0 or Google Pixel 7-13.0). This blocks two parallel test licenses in your App Automate subscription plan, one for each shard.