Run Cucumber-based Espresso tests

App Automate supports running your Cucumber-based Espresso test. The cucumberOptions JSON object allows your Cucumber-based Espresso tests to run similar to any other Espresso test.

Note: If you have added multiple examples within a Scenario outline in your feature file, all the examples will run as a single scenario on BrowserStack. These examples will have combined debugging logs under one Scenario on your App Automate dashboard and in REST API response.

In this guide, you’ll learn how to:

Filter and run test cases
Generate test reports
Use test sharding

Filter test cases

By default, all your test cases in your Cucumber-based Espresso test suite are executed on BrowserStack. However, you can filter test cases using the cucumberOptions JSON object to:

Run a quick smoke test to ensure the sanity of critical functionality.
Run only those test cases that are relevant to a new bug fix.
Split a large test suite into multiple shards and run them in parallel.

You can filter and run test cases using the following ways:

Filter test cases using feature file path

Specify the path of your feature file in the features parameter of the cucumberOptions JSON object to run feature files or a particular scenario written in a feature file.

Options available within the filter functionality are as follows:

Run all available feature files in a directory
Run specific feature file(s)
Run test scenarios written within certain lines of the feature file

Run all available feature files of a directory

You can run all the feature files that are located inside a given directory.

Specify the path of the directory in the features parameter of the cucumberOptions JSON object when running the execute build cURL command.

The following example cURL command uses the features parameter to run all features located inside the features/addition/decimal and features/multiply directories:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": { "features": ["features/addition/decimal", "features/multiplication"] }, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

Run test cases of specific feature file(s)

You can run certain test cases written in feature files.

Specify the path of the feature file in the features parameter of the cucumberOptions JSON object when running the execute build cURL command.

The following example cURL command uses the features parameter to run the compose.feature feature file located inside the features/extra directory:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": { "features": ["features/addition/decimal/add.feature", "features/multiplication/multiply.feature"] }, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

Run scenarios written within certain lines of feature file

You can run certain scenarios that are written within certain lines of a feature file.

Specify the name and line numbers of the feature file in the features parameter of the cucumberOptions JSON object when running the execute build cURL command.

The following example cURL command uses the features parameter to run a scenario written between line number 4 to 8 of the features/addition/decimal/add.feature file:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": { "features": ["features/addition/decimal/add.feature:4:8"] }, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

Filter test cases using tags

Filter and run test cases that are annotated with specific tags in the code.

Specify name of the tag in the tag parameter of the cucumberOptions JSON object when running the execute build cURL command.

The following example cURL command runs all feature files annotated with the @Sum tag:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": {"tags":"@Sum"}, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

Filter test cases using scenario name

Specify the regex of the test scenario name in the name parameter of the cucumberOptions JSON object when running the execute build cURL command.

The following example cURL command uses the name parameter to run those tests where the scenario name contains the word addition:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": {"name":"addition"}, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

Generate test reports

A test report gives an overview of the features steps and their result. Test reports of your Cucumber-based Espresso tests are generated using the plugins parameter of the cucumberOptions JSON object when running the execute build cURL command.

The following table lists the values that the plugins parameter accepts to generate a test report:

Parameter	Description	Values
`plugins`	Generate test report of the Cucumber-based Espresso tests.	`pretty`: Generate test report in the Pretty format `json`: Generate test report in the JSON format `junit`: Generate test report in the XML format `html`: Generate test report in the HTML format

Complete the following steps to generate and download test report:

Specify format of test report in command
Download the generated test report

Step 1: Specify format of test report in command

Specify the format of the report in the plugins parameter of the cucumberOptions JSON object, as shown in the example execute build cURL command:

  curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
  -X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
  -d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": {"plugins": ["junit"]}, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
  -H "Content-Type: application/json"

To generate test report in more than one format, specify the report format in a JSON array, as shown in the following example cURL command:

  curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
  -X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
  -d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": {"plugins": ["html","junit","pretty"]},, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
  -H "Content-Type: application/json"

Step 2: Download the generated test report

Specify a name for the report file, and set your build and session ID in the following cURL command:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" -X GET "https://api-cloud.browserstack.com/app-automate/espresso/v2/builds/{build_id}/sessions/{session_id}/assets" >> <filename>.zip

This command downloads a compressed file of the test report to your machine.

Generate reports for Android <= 10.0

To generate test/coverage reports for Android OS 10 and lower, make the following additional changes to the manifest.xml file:

- Add android.permission.WRITE_EXTERNAL_STORAGE
- Add android.permission.READ_EXTERNAL_STORAGE
- Set android:requestLegacyExternalStorage="true"

Use test sharding

All test cases in your Cucumber-based Espresso tests that are run using cucumberOptions are, by default, executed sequentially.

However, instead of running your test suite sequentially, you can split those test cases into different groups called shards. Each shard is then executed in parallel on one or more devices, decreasing the test execution time.

To use sharding, add the shards object to the execute build cURL command and set the strategy parameter to cucumberOptions. You can also add filters in the values parameter.

An example JSON structure to add to the shards object is as follows:

{
  "shards": {
    "numberOfShards": 2,
    "mapping": [{
      "name": "Shard 1",
      "strategy": "cucumberOptions",
      "values": {
        "tags": "TAG_EXPRESSION",
        "name": "REGEXP",
        "features": "FEATURE_FILE_PATH"
      }
    }, {
      "name": "Shard 2",
      "strategy": "cucumberOptions",
      "values": {
        "tags": "TAG_EXPRESSION",
        "name": "REGEXP",
        "features": "FEATURE_FILE_PATH"
      }
    }]
  },
  "devices": ["Samsung Galaxy S20-10.0", "Google Pixel 3-10.0"],
  "app": "<app-id>",
  "testSuite": "<test-suite-id>"
}

The following example cURL command splits the test cases into two shards, filters test cases using the tags filter, and runs the test on Samsung Galaxy S20-10.0 and Google Pixel 3-10.0 devices:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{ "shards": { "numberOfShards": 2, "mapping": [{ "name": "Shard 1", "strategy": "cucumberOptions", "values": { "tags":"@first" }}, { "name": "Shard 2", "strategy": "cucumberOptions", "values": { "tags":"@second" }}]}, "devices": [ "Samsung Galaxy S20-10.0", "Google Pixel 3-10.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

Check out the test sharding section to learn more about sharding.

Did this page help you?

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback

Is this page helping you?

Yes

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback!