Test execution count for billing in Test Reporting & Analytics

Learn how counts tests for billing purposes.

Test Reporting & Analytics uses the number of test executions as the metric for billing.

For Test Driven Development (TDD), Test Reporting & Analytics counts each it block in Test Driven Development (TDD) as a separate test. Similarly, Test Reporting & Analytics counts each scenario as an individual test in Behavior Driven Development (BDD).

Test Reporting & Analytics provides you with four options to handle the global hooks like beforeAll and afterAll:

• Ignore all global hooks.
• Show only failed beforeAll hooks.
• Show only failed hooks.
• Include all global hooks.

By default, only failed beforeAll hooks are included in the test listing.

Depending on your choice, hooks are included or excluded as tests for billing purposes.

You can verify if an entry is counted as a test by checking if Test Reporting & Analytics displays it as a separate line item in the test listing.

Illustrating test execution count with an example

Test Reporting & Analytics bills the total number of non-BrowserStack test executions by multiplying the number of non-BrowserStack tests by the number of times they run across various browser and device combinations.

The following sample table illustrates how the test executions are counted:

You can see that the total number of non-BrowserStack test executions considered for billing (last column) is calculated by multiplying the number of different types of tests (column D) with the corresponding number of build runs (column E) and adding them up.

Non-BrowserStack test executions

Any test execution run outside the BrowserStack infrastructure is a non-BrowserStack test execution. These include local test executions and test executions run on other cloud service providers. Non-BrowserStack test executions can be of any type like unit tests, regression tests, and API tests.

If you exceed the limit of non-BrowserStack test executions, the builds will still be created on the dashboard, but they’ll be locked. To view the build data, you will need to update your plan.