Use Test Sharding

Parallel testing enables you to reduce the overall test execution times, resulting in improved build times and faster releases. Test Sharding enables you to test at scale on BrowserStack.

Introduction

By default, all test cases in your Espresso test-suite are executed sequentially. Instead of running your test-suite sequentially, you can split test cases into different groups called shards. Each shard can then be executed in parallel on one or more devices. This is especially helpful when your test-suite has a large number of test cases.

For example, if your test-suite has 500 test cases. Assume each test case takes about a minute to execute. If the test-suite is executed sequentially, the overall execution time will be close to 500 mins. However, if you split your test-suite into 10 shards, each shard will consist of 50 test cases. If each shard is executed in parallel, the total test execution time will be down to 50 mins.

To enable test sharding, you need to configure the shards parameter in the REST API request to start Espresso test execution.

REST API endpoint :

POST /app-automate/espresso/v2/build
Parameter Description Value
shards Split the test-suite into multiple shards and execute them in parallel. An object (key-value pairs) to specify the number of shards and configure its behaviour. Refer to REST API for detailed reference.

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2}, "devices": ["Google Pixel 3-9.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://j3c874f21852ba57957a3fdc33f47514288c4ba4"}' \
-H "Content-Type: application/json"

In this example, the test-suite will be split into 2 shards, and each of those shards will execute in parallel on Google Pixel 3-9.0. This will block 2 parallel tests licenses in your App Automate subscription plan.

Sharding strategy

There are multiple strategies to control how you want to split the test-suite into different shards.

1. Use auto strategy

This is simplest way to use test sharding. The test-suite is automatically split into the desired number of shards. Each shard will have approximately same number of test cases. We rely on the AndroidJUnitRunner shard feature to implement this behaviour. All the shards will be executed in parallel on all the devices specified in the devices parameter.

REST API endpoint :

POST /app-automate/espresso/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 3-10.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be automatically split into 2 groups (i.e. shards). Both the shards will execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 3-10.0. This will block 4 parallel tests licenses in your App Automate subscription plan.

2. Use package strategy

Using this strategy, you can group test cases by their fully qualified Java package names (e.g. com.foo.login) into different shards. All the shards will be executed in parallel on the devices specified in the devices parameter.

REST API endpoint :

POST /app-automate/espresso/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "package", "values": ["com.foo.login", "com.foo.logout"]}, {"name": "Shard 2", "strategy": "package", "values": ["com.foo.dashboard"]}]}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 3-10.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be split into 2 groups (i.e. shards). The first shard will consist of test cases belonging to com.foo.login and com.foo.logout packages. The second shard will consist of test cases belong to the com.foo.dashboard package. Both the shards will execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 3-10.0. This will block 4 parallel tests licenses in your App Automate subscription plan.

3. Use class strategy

Using this strategy, you can group test cases by their fully qualified Java class names (e.g. com.foo.login.user) into different shards. All the shards will be executed in parallel on the devices specified in the devices parameter.

REST API endpoint :

POST /app-automate/espresso/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "class", "values": ["com.foo.login.user", "com.foo.login.admin"]}, {"name": "Shard 2", "strategy": "class", "values": ["com.foo.logout.user"]}]}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 3-10.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be split into 2 groups (i.e. shards). The first shard will consist of test cases belonging to com.foo.login.user and com.foo.login.admin classes. The second shard will consist of test cases belong to the com.foo.logout.user class. Both the shards will execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 3-10.0. This will block 4 parallel tests licenses in your App Automate subscription plan.

4. Use annotation strategy

Using this strategy, you can group test cases by annotations (e.g. com.foo.login.SmokeSuite) into different shards. All the shards will be executed in parallel on the devices specified in the devices parameter.

REST API endpoint :

POST /app-automate/espresso/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "annotation", "values": ["com.foo.login.SmokeSuite"]}, {"name": "Shard 2", "strategy": "annotation", "values": ["com.foo.login.RegressionSuite"]}]}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 3-10.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be split into 2 groups (i.e. shards). The first shard will consist of test cases with com.foo.login.SmokeSuite annotation. The second shard will consist of test cases belong to the com.foo.login.RegressionSuite annotation. Both the shards will execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 3-10.0. This will block 4 parallel tests licenses in your App Automate subscription plan.

5. Use size strategy

Using this strategy, you can group test cases by specific test size i.e tests annotated with @SmallTest, @MediumTest or @LargeTest (e.g. small/medium/large) into different shards. All the shards will be executed in parallel on the devices specified in the devices parameter.

REST API endpoint :

POST /app-automate/espresso/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "size", "values": ["small", "medium"]}, {"name": "Shard 2", "strategy": "size", "values": ["large"]}]}, "devices": ["Samsung Galaxy S20-10.0","Google Pixel 3-10.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be split into 2 groups (i.e. shards). The first shard will consist of test cases annotated with @SmallTest and @MediumTest. The second shard will consist of test cases annotated with @LargeTest. Both the shards will execute in parallel on Samsung Galaxy S20-10.0 and Google Pixel 3-10.0. This will block 4 parallel tests licenses in your App Automate subscription plan.

Device selection strategy

If you don’t want each of the shards to execute on all specified devices, you can set the deviceSelection attribute inside the shards parameter to any value. This will ensure that each shard will execute on any randomly selected device from the list of devices specified in the devices parameter. By default, its value is set to all.

Parameter Description Values
deviceSelection Run each shard on all devices or any randomly selected device from the list of devices specified in the devices parameter. all , any
Default: all

all​ : Each shard will run on all specified devices.
any​ : Each shard will run on any one randomly selected device.

REST API endpoint :

POST /app-automate/espresso/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"shards": {"numberOfShards": 2, "deviceSelection": "any", "mapping": [{"name": "Shard 1", "strategy": "class", "values": ["com.foo.test1", "com.foo.test2"]}, {"name": "Shard 2", "strategy": "class", "values": ["com.foo", "com.foo.login"]}]}, "devices": ["Samsung Galaxy S8-7.0", "Google Pixel 3-10.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be split into 2 groups (i.e. shards). Both the shards will execute in parallel, but each shard will execute on a single device randomly selected from the list (i.e. Samsung Galaxy S20-10.0 or Google Pixel 3-10.0). This will block 2 parallel tests licenses in your App Automate subscription plan (one for each shard).

We're sorry to hear that. Please share your feedback so we can do better







Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback

Is this page helping you?

Yes
No

We're sorry to hear that. Please share your feedback so we can do better







Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback!

Talk to an Expert
Talk to an Expert