Get started with Screen Reader Automation

Learn how to set up and use BrowserStack’s Screen Reader Automation for Android.

Screen Reader Automation for Android allows you to automate accessibility testing by simulating screen reader interactions, capturing screen reader output, and validating accessibility metadata directly within your test scripts. This guide will help you get started with setting up and using Screen Reader Automation.

This guide covers the following topics:

• How screen reader automation works
• Implement screen reader automation
  • Enable or disable TalkBack
  • Simulate screen reader gestures
    • List of supported gestures
  • Capture screen reader spoken output
• Add accessibility assertions

How screen reader automation works

BrowserStack provides the browserstack_executor command, which enables you to have granular control over the screen reader environment within your tests. It is a custom extension that allows your test scripts to send commands to control the screen reader on real BrowserStack devices.

This feature provides the following capabilities:

• Programmatic Control: Use the browserstack_executor command with the screenReader action to enable and disable TalkBack directly from your test scripts.
• Gesture Simulation: Use the browserstack_executor command with the screenReaderGesture action to programmatically simulate common accessibility gestures like swipe left, swipe right, or double-tap.
• Real Output Capture: Use the browserstack_executor command with the screenReaderSpokenDescription action to extract the screen reader’s actual spoken output as plain text.
• Assertions: Use the captured spoken output to add assertions in your test scripts, ensuring that your app meets accessibility standards. For more information, see Add accessibility assertions in Automated tests.

Implement screen reader automation

Integrate the browserstack_executor command into your test scripts to perform the following actions:

1. Enable or disable TalkBack.
2. Simulate screen reader gestures.
3. Capture screen reader spoken output.

Enable or disable TalkBack

Use the screenReader action to enable or disable TalkBack.

• To enable TalkBack:

  driver.execute_script("browserstack_executor: {\"action\":\"screenReader\",\"arguments\": {\"enable\" : \"true\"}}");
  

• To disable TalkBack:

  driver.execute_script("browserstack_executor: {\"action\":\"screenReader\",\"arguments\": {\"enable\" : \"false\"}}");
  

Simulate screen reader gestures

Use the screenReaderGesture action to simulate common screen reader gestures, such as navigate, activate, or scroll.

• To simulate a navigate next gesture:

  driver.execute_script("browserstack_executor: {\"action\":\"screenReaderGesture\",\"arguments\": {\"gesture\" : \"navigate_next\"}}");
  

• To simulate an activate gesture:

  driver.execute_script("browserstack_executor: {\"action\":\"screenReaderGesture\",\"arguments\": {\"gesture\" : \"activate_item\"}}");
  

• To simulate a scroll down gesture:

  driver.execute_script("browserstack_executor: {\"action\":\"screenReaderGesture\",\"arguments\": {\"gesture\" : \"scroll_down\"}}");
  

List of supported gestures

The following TalkBack gestures are supported for screen reader automation:

Capture screen reader spoken output

Use the screenReaderSpokenDescription action and specify the resource ID of an element to capture the spoken output of that element. If you do not specify a resource ID, the command provides the spoken output for all the elements captured by the screen reader.

To capture the spoken output of a specific UI element:

driver.execute_script("browserstack_executor: {\"action\":\"screenReaderSpokenDescription\",\"arguments\": {\"resourceId\" : \"<your-element-resource-id>\"}}")

The command returns the spoken output as a string, which you can then log, use for assertions, or save to a file for offline review.

Add accessibility assertions

With the captured screen reader data, you can add powerful assertions to your test scripts. This enables you to verify critical accessibility aspects:

• Focusability: Verify that all relevant UI elements receive focus and are accessible to the screen reader.
• Spoken Output Verification: Check that elements have the correct accessibility metadata, such as labels, roles, and hints, and that the actual screen reader output for an element matches your expected accessibility label or announcement.
• Traversal Order: Record the screen reader traversal sequence and ensure that the focus order of UI elements follows a logical reading flow by comparing it to the DOM structure or visual test.

Enable assertions

You can enable assertions in your test scripts by using the browserstack_executor command with the screenReaderSpokenDescription action to capture the spoken output of a specific UI element and verify it against your expected values.

To enable assertions:

driver.execute_script("browserstack_executor: {\"action\":\"screenReaderSpokenDescription\",\"arguments\": {\"resourceId\" : \"<your-element-resource-id>\"}}");

Example assertion

Spoken Output Verification:
The following example demonstrates how to capture the spoken output of a UI element and assert that it matches the expected spoken description:

print("Getting spoken text of element\n")
    out = driver.execute_script("browserstack_executor: {\"action\":\"screenReaderSpokenDescription\",\"arguments\": {\"resourceId\" : \"<your-element-resource-id>\"}}");
    expected_description = "Master Switch, ON"
    actual_description = out['spoken_description'][0]

    if actual_description == expected_description
      puts "Assertion Passed: Spoken description is correct."
    else
      raise "Assertion Failed: Expected '#{expected_description}', but got '#{actual_description}'"
    end