No Result Found
Connect & Get help from fellow developers on our Discord community. Ask the CommunityAsk the Community

Specify testing scope in automated accessibility tests

Define the scope for automated accessibility testing in your test suite.

In some cases, you may want to run accessibility tests only on specific device combinations or for specific test cases. You can configure your test suite to limit the scope of testing to only those devices and test cases.

Test on specific device combinations

In your configuration file or cURL command, you can include or exclude specific device combinations to control your testing scope.

For Android devices, automated app accessibility tests are supported on version 11 and above.

For test runners that use YAML, set the accessibility flag on device combination that you want to include.

userName: YOUR_USERNAME
accessKey: YOUR_ACCESS_KEY
...
accessibility: true # root level flag to enable accessibility testing
platforms:
  - platformName: android
    deviceName: Samsung Galaxy S22
    platformVersion: 12.0
    accessibility: true # Testing will be conducted on this device combination
  - platformName: android
    deviceName: Google Pixel 6
    platformVersion: 13.0
    accessibility: false # Testing will not be conducted on this device combination
...

For test runners that use JavaScript, set the accessibility flag on the device combination that you want to include.

exports.config = {
  user: process.env.BROWSERSTACK_USERNAME || 'YOUR_USERNAME',
  key: process.env.BROWSERSTACK_ACCESS_KEY || 'YOUR_ACCESS_KEY',
  
  hostname: 'hub.browserstack.com',
  services: [
    [
      'browserstack',
      {
        accessibility: true, // root level flag to enable accessibility testing
        ...
      }
    ]
  ],
  capabilities: [{
    'bstack:options': {
      deviceName: 'Samsung Galaxy S22',
      osVersion: "12.0",
      accessibility: true //Testing will be conducted on this device combination
    }
  }, {
    'bstack:options': {
      deviceName: 'Google Pixel 6',
      osVersion: "13.0",
      accessibility: false //Testing will not be conducted on this device combination
    }
  }],
  ...
};

For Espresso tests, specify the device combinations in the cURL command.

You can’t exclude individual devices from accessibility tests. Tests will run on all devices in the devices array.

curl -u "USERNAME:ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d "{
    \"app\": \"bs://$app_id\",
    \"testSuite\": \"bs://$test_suite_id\",
    \"devices\": ["Samsung S24-14.0"], ["Google Pixel 8-14.0"], ["Samsung S21-11.0"]
    \"accessibility\": true,
    \"accessibilityOptions\": {
        ...
        }
    }" \
-H "Content-Type: application/json"

Test specific test cases

You can tag test cases to control which ones are included or excluded during accessibility testing. The tagging method depends on the test runner you use.

Define tags for the tests you want to run in your test case file.


public class FirstTest extends AppiumTest {
    private void navigateToSearchResults() throws Exception {
        // Common setup steps for both tests
        WebElement skipButton = (WebElement) new WebDriverWait(driver, Duration.ofSeconds(30)).until(
            ExpectedConditions.presenceOfElementLocated(AppiumBy.id("org.wikipedia.alpha:id/fragment_onboarding_skip_button")));
        skipButton.click();

        WebElement searchElement = (WebElement) new WebDriverWait(driver, Duration.ofSeconds(30)).until(
            ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("Search Wikipedia")));
        searchElement.click();

        WebElement insertTextElement = (WebElement) new WebDriverWait(driver, Duration.ofSeconds(30)).until(
            ExpectedConditions.elementToBeClickable(AppiumBy.id("org.wikipedia.alpha:id/search_src_text")));
        insertTextElement.sendKeys("BrowserStack");

        Thread.sleep(5000);
    }

    @Test(groups = "search")
    public void testSearchFunctionality() throws Exception {
        navigateToSearchResults();
        List<WebElement> allProductsName = driver.findElements(AppiumBy.className("android.widget.TextView"));
        Assert.assertTrue(allProductsName.size() > 0);
    }

    @Test(groups = "accessibility")
    public void testAccessibilityScan() throws Exception {
        navigateToSearchResults();
        Map<String, Object> summary = AccessibilityUtils.getResultsSummary(driver);
        System.out.println("=== Accessibility Summary ===");
        for (Map.Entry<String, Object> entry : summary.entrySet()) {
            String key = entry.getKey();
            Object value = entry.getValue();
            if (value instanceof Map) {
                System.out.println(key + ":");
                Map<?, ?> nestedMap = (Map<?, ?>) value;
                for (Map.Entry<?, ?> nestedEntry : nestedMap.entrySet()) {
                    System.out.println("  " + nestedEntry.getKey() + " = " + nestedEntry.getValue());
                }
            } else {
                System.out.println(key + " = " + value);
            }
        }
    }
}


describe('Search Wikipedia Functionality', () => {
  it("start", async () => {
    var skipButton = await $('android=new UiSelector().resourceId("org.wikipedia.alpha:id/fragment_onboarding_skip_button")');
    await skipButton.waitForDisplayed({ timeout: 30000 });
    await skipButton.click();
  });

  it("mid", async () => {
    var searchSelector = await $(`~Search Wikipedia`);
    await searchSelector.waitForDisplayed({ timeout: 30000 });
    await searchSelector.click();
  });

  it("end", async () => {
    var insertTextSelector = await $('android=new UiSelector().resourceId("org.wikipedia.alpha:id/search_src_text")');
    await insertTextSelector.waitForDisplayed({ timeout: 30000 });
    await insertTextSelector.addValue("BrowserStack");
    await browser.pause(5000);
  });
});

Include or exclude test cases from the testing scope based on their tag, annotation, or groups. These need to be specified under the accessibilityOptions key in your configuration file.

...
accessibilityOptions:
  wcagVersion: wcag21aa
  includeTagsInTestingScope: ["accessibility"] # This test case will be included in the test.
  excludeTagsInTestingScope: ["search"] # This test case will be excluded from the test.
  scannerProcessingTimeout: 10
  includeIssueType:
    bestPractice: false
...  

...
accessibilityOptions:{
    wcagVersion: 'wcag21aaa',
    includeTagsInTestingScope: ["mid", "end"], // These test cases will be included in the test.
    excludeTagsInTestingScope: ["start"] // This test case will be excluded from the test.
    scannerProcessingTimeout: 10
    includeIssueType: {
            bestPractice: false
            }
  },
...

Access the report in your project folder on the App Accessibility dashboard.

Additionally, here are some important considerations for the inclusion & exclusion list:

  • If no tags are found in either list, testing will take place on all test cases.
  • If tags are found in both lists, the final set of test cases will be identified for testing by removing the exclusion list test cases from the inclusion list test cases.

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked





Thank you for your valuable feedback

  • ON THIS PAGE

    Is this page helping you?

    Yes
    No

    We're sorry to hear that. Please share your feedback so we can do better

    Contact our Support team for immediate help while we work on improving our docs.

    We're continuously improving our docs. We'd love to know what you liked





    Thank you for your valuable feedback!

    Talk to an Expert
    Download Copy Check Circle