Accessibility testing using the axe-core library

A guide to running accessibility tests in BrowserStack Automate using the axe-core open source library

This document provides you with the step-by-step process to verify if your website follows the WCAG and other guidelines (as supported by axe-core library), ensuring your website is accessible to all types of users on the Internet. Axe is a fast and lightweight accessibility testing tool that checks the entire document against the rules and generates a report with all violations, passes, etc.

In this guide, you will learn:

How axe-core library is used with BrowserStack Automate to achieve Accessibility testing
What are the components of the generated report

Pre-requisites

Before you can integrate your tests with BrowserStack and generate reports using the axe-core library, ensure that the following pre-requisites are complete:

Download the axe.min.js file and save it in your local system. You will use this file in the sample code snippets.

Run your first accessibility test

The following sample script opens the file you downloaded in the pre-requisite step and injects the axe.min.js script in the DOM of the website under test. After that, using a Javascript Executor, the accessibility checks are invoked using the axe-core library. The generated report is saved in a JSON file as per the path provided in the script.

You can also write assertions in your script based on the violations in the report.json and mark your test passed or failed based on these assertions.

Selenium 4 W3C
Selenium Legacy JSON

import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.json.Json;
import org.openqa.selenium.MutableCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.io.File;
import java.io.FileWriter;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import com.google.gson.Gson;
import com.google.gson.JsonObject;

public class test{
	public static final String AUTOMATE_USERNAME = "YOUR_USERNAME";
	public static final String AUTOMATE_ACCESS_KEY = "YOUR_ACCESS_KEY";
	public static final String URL = "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub";
	public static void main(String[] args) throws Exception {
	MutableCapabilities capabilities = new MutableCapabilities();
	capabilities.setCapability("browserName", "Chrome");
	capabilities.setCapability("browserVersion", "103.0");
	HashMap<String, Object> browserstackOptions = new HashMap<String, Object>();
	browserstackOptions.put("os", "Windows");
	browserstackOptions.put("osVersion", "10");
	browserstackOptions.put("projectName", "BStack Accessibility Test");
	browserstackOptions.put("buildName", "BStack Accessibility Sample Build");
	capabilities.setCapability("bstack:options", browserstackOptions);


		 WebDriver driver = new RemoteWebDriver(new URL(URL), caps);
		 driver.get("https://www.google.com");

		 JavascriptExecutor jse = (JavascriptExecutor)driver;
		 Path path = Paths.get("path/to/axe.min.js");

		 String content = new String(Files.readAllBytes(path));
		 jse.executeScript(content);

		 File output = new File("path/to/report.json");
		 FileWriter writer = new FileWriter(output);
		 String result = String.valueOf(jse.executeAsyncScript("var callback = arguments[arguments.length - 1]; axe.run().then(results => callback(results));"));
         Gson g = new Gson();
         String p = g.toJson(result);
         writer.write(p);
		 writer.flush();
		 writer.close();
		 driver.quit();
	}
}

const webdriver = require('selenium-webdriver');
const fs = require('fs')

// Input capabilities
	var capabilities = {
		'bstack:options' : {
			"os" : "Windows",
			"osVersion" : "10",
			"projectName" : "BStack Accessibility Test",
			"buildName" : "BStack Accessibility Sample Build",
			"userName" : "YOUR_USERNAME",
			"accessKey" : "YOUR_ACCESS_KEY",
		},
		"browserName" : "Chrome",
		"browserVersion" : "103.0",
	}
async function runAccessibilityTest () {
  let driver = new webdriver.Builder()
    .usingServer('https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub')
    .withCapabilities(capabilities)
    .build();
  await driver.get("https://www.google.com");
  const data = await fs.readFileSync('path/to/axe.min.js', 'utf8')
  await driver.executeScript(data.toString());
  let result = await driver.executeAsyncScript('var callback = arguments[arguments.length - 1];axe.run().then(results => callback(results))');
  await fs.writeFileSync('path/to/report.json', JSON.stringify(result));
  await driver.quit();
}
runAccessibilityTest();

using System;
using System.IO;
using Newtonsoft.Json;
using OpenQA.Selenium.Remote;
namespace SeleniumTest
{
    class Program
    {
        static readonly string axe = @"/path/to/axe.min.js";
        static readonly string report = @"/path/where/you/want/to/save/report.json";
        static void Main(string[] args)
        {
            RemoteWebDriver driver;
            ChromeOptions capabilities = new ChromeOptions();
            capabilities.BrowserVersion = "103.0";
            Dictionary<string, object> browserstackOptions = new Dictionary<string, object>();
            browserstackOptions.Add("os", "Windows");
            browserstackOptions.Add("osVersion", "10");
            browserstackOptions.Add("projectName", "BStack Accessibility Test");
            browserstackOptions.Add("buildName", "BStack Accessibility Sample Build");
            browserstackOptions.Add("userName", "YOUR_USERNAME");
            browserstackOptions.Add("accessKey", "YOUR_ACCESS_KEY");
            browserstackOptions.Add("browserName", "Chrome");
            capabilities.AddAdditionalOption("bstack:options", browserstackOptions);

            driver = new RemoteWebDriver(
                new Uri("https://hub-cloud.browserstack.com/wd/hub/"), capability
                );
            driver.Navigate().GoToUrl("https://www.google.com/");
            if (File.Exists(axe))
            {
                string axeCode = File.ReadAllText(axe);
                driver.ExecuteScript(axeCode);
                IJavaScriptExecutor javascriptExecutor = (IJavaScriptExecutor) driver;
                string result = JsonConvert.SerializeObject(javascriptExecutor.ExecuteAsyncScript("var callback = arguments[arguments.length - 1]; axe.run().then(results => callback(results))"), Formatting.Indented);
                File.WriteAllText(report, result);
            }
            driver.Quit();
        }
    }
}

<?php
  require_once('vendor/autoload.php');
  use Facebook\WebDriver\Remote\RemoteWebDriver;
  use Facebook\WebDriver\WebDriverBy;
    $caps = array(
    'bstack:options' => array(
      "os" => "Windows",
      "osVersion" => "10",
      "projectName" => "BStack Accessibility Test",
      "buildName" => "BStack Accessibility Sample Build",
    ),
    "browserName" => "Chrome",
    "browserVersion" => "103.0",
    )

  $web_driver = RemoteWebDriver::create(
    "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
    $caps
  );
  $web_driver->get("https://www.google.com");

  $axe_script = fopen("path/to/axe.min.js", "r");
  $web_driver->executeScript(fread($axe_script,filesize("path/to/axe.min.js")));
  fclose($axe_script);
  $result = $web_driver->executeAsyncScript('var callback = arguments[arguments.length - 1];axe.run().then(results => callback(results))');
  $file = fopen("path/to/report.json", "w");
  // $reportJSON = json_encode($result);
  fwrite($file,json_encode($result, JSON_PRETTY_PRINT));
  fclose($file);
  $web_driver->quit();
?>

from selenium import webdriver
import json

desired_cap = {
	'bstack:options' : {
		"os" : "Windows",
		"osVersion" : "10",
		"projectName" : "BStack Accessibility Test",
		"buildName" : "BStack Accessibility Sample Build",
	},
	"browserName" : "Chrome",
	"browserVersion" : "103.0",
}

driver = webdriver.Remote(
    desired_capabilities=desired_cap,
    command_executor='https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub'
    )
driver.get("https://www.microsoft.com")
axe_script = open('./axe.min.js', 'r')
driver.execute_script(axe_script.read())
axe_script.close()
result = driver.execute_async_script('var callback = arguments[arguments.length - 1];axe.run().then(results => callback(results))')
file = open("./report.json", "w")
file.write(json.dumps(eval(str(result))))
file.close()
driver.quit()

require	'rubygems'
require	'selenium-webdriver'
require 'json'

#Input Capabilities
capabilities = {
	'bstack:options' => {
		"os" => "Windows",
		"osVersion" => "10",
		"projectName" => "BStack Accessibility Test",
		"buildName" => "BStack Accessibility Sample Build",
	},
	"browserName" => "Chrome",
	"browserVersion" => "103.0",
}


driver = Selenium::WebDriver.for(:remote,
	:url => "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
	:desired_capabilities => caps)

	# Searching for 'BrowserStack' on google.com
	driver.navigate.to "https://www.google.com"
	axe_script = open('path/to/axe.min.js', 'r')
	driver.execute_script(axe_script.read())
	axe_script.close()
	result = driver.execute_async_script('var callback = arguments[arguments.length - 1];axe.run().then(results => callback(results))')
	file = open("path/to/report.json", "w")
	file.write(result.to_json)
	file.close()
	driver.quit

You can also conduct Accessibility testing of your websites on browsers where High Contrast Mode is enabled by setting BrowserStack’s custom capability high_contrast to true. Check out high contrast mode to learn more.

import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.json.Json;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.io.File;
import java.io.FileWriter;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import com.google.gson.Gson;
import com.google.gson.JsonObject;

public class test{
	public static final String AUTOMATE_USERNAME = "YOUR_USERNAME";
	public static final String AUTOMATE_ACCESS_KEY = "YOUR_ACCESS_KEY";
	public static final String URL = "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub";
	public static void main(String[] args) throws Exception {
		 DesiredCapabilities caps = new DesiredCapabilities();
		 caps.setCapability("os_version", "10");
		 caps.setCapability("browser", "chrome");
		 caps.setCapability("browser_version", "latest");
		 caps.setCapability("os", "Windows");
		 caps.setCapability("name", "Accessibility Test - Google"); // test name
		 caps.setCapability("build", "Java Accessibility Test Sample Build"); // CI/CD job or build name
		 WebDriver driver = new RemoteWebDriver(new URL(URL), caps);
		 driver.get("https://www.google.com");

		 JavascriptExecutor jse = (JavascriptExecutor)driver;
		 Path path = Paths.get("path/to/axe.min.js");

		 String content = new String(Files.readAllBytes(path));
		 jse.executeScript(content);

		 File output = new File("path/to/report.json");
		 FileWriter writer = new FileWriter(output);
		 String result = String.valueOf(jse.executeAsyncScript("var callback = arguments[arguments.length - 1]; axe.run().then(results => callback(results));"));
         Gson g = new Gson();
         String p = g.toJson(result);
         writer.write(p);
		 writer.flush();
		 writer.close();
		 driver.quit();
	}
}

const webdriver = require('selenium-webdriver');
const fs = require('fs')

// Input capabilities
const capabilities = {
  'browserName': 'chrome',
  'browserVersion': 'latest',
  'os': 'windows',
  'os_version': '10',
  'build': 'Accessibility Test Sample Build',
  'name': 'Accessibility test - Google'
}
async function runAccessibilityTest () {
  let driver = new webdriver.Builder()
    .usingServer('https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub')
    .withCapabilities(capabilities)
    .build();
  await driver.get("http://www.google.com");
  const data = await fs.readFileSync('path/to/axe.min.js', 'utf8')
  await driver.executeScript(data.toString());
  let result = await driver.executeAsyncScript('var callback = arguments[arguments.length - 1];axe.run().then(results => callback(results))');
  await fs.writeFileSync('path/to/report.json', JSON.stringify(result));
  await driver.quit();
}
runAccessibilityTest();

using System;
using System.IO;
using Newtonsoft.Json;
using OpenQA.Selenium.Remote;
namespace SeleniumTest
{
    class Program
    {
        static readonly string axe = @"/path/to/axe.min.js";
        static readonly string report = @"/path/where/you/want/to/save/report.json";
        static void Main(string[] args)
        {
            RemoteWebDriver driver;
            OpenQA.Selenium.Chrome.ChromeOptions capability = new OpenQA.Selenium.Chrome.ChromeOptions();
            capability.AddAdditionalCapability("os_version", "10", true);
            capability.AddAdditionalCapability("browser", "Chrome", true);
            capability.AddAdditionalCapability("browser_version", "latest", true);
            capability.AddAdditionalCapability("os", "Windows", true);
            capability.AddAdditionalCapability("name", "Accessibility Test - Google", true); // test name
            capability.AddAdditionalCapability("build", "C-sharp Accessibility Test Sample Build", true); // CI/CD job or build name
            capability.AddAdditionalCapability("browserstack.user", "YOUR_USERNAME", true);
            capability.AddAdditionalCapability("browserstack.key", "YOUR_ACCESS_KEY", true);
            driver = new RemoteWebDriver(
                new Uri("https://hub-cloud.browserstack.com/wd/hub/"), capability
                );
            driver.Navigate().GoToUrl("https://www.google.com/");
            if (File.Exists(axe))
            {
                string axeCode = File.ReadAllText(axe);
                driver.ExecuteScript(axeCode);
                IJavaScriptExecutor javascriptExecutor = (IJavaScriptExecutor) driver;
                string result = JsonConvert.SerializeObject(javascriptExecutor.ExecuteAsyncScript("var callback = arguments[arguments.length - 1]; axe.run().then(results => callback(results))"), Formatting.Indented);
                File.WriteAllText(report, result);
            }
            driver.Quit();
        }
    }
}

<?php
  require_once('vendor/autoload.php');
  use Facebook\WebDriver\Remote\RemoteWebDriver;
  use Facebook\WebDriver\WebDriverBy;
  $caps = array(
    "os_version" => "10",
    "browser" => "Chrome",
    "browser_version" => "latest",
    "os" => "Windows",
    "name" => "Accessibility testing", // test name
    "build" => "Accessibility Testing Sample Build" // CI/CD job or build name
  );
  $web_driver = RemoteWebDriver::create(
    "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
    $caps
  );
  $web_driver->get("https://www.google.com");

  $axe_script = fopen("path/to/axe.min.js", "r");
  $web_driver->executeScript(fread($axe_script,filesize("path/to/axe.min.js")));
  fclose($axe_script);
  $result = $web_driver->executeAsyncScript('var callback = arguments[arguments.length - 1];axe.run().then(results => callback(results))');
  $file = fopen("path/to/report.json", "w");
  // $reportJSON = json_encode($result);
  fwrite($file,json_encode($result, JSON_PRETTY_PRINT));
  fclose($file);
  $web_driver->quit();
?>

from selenium import webdriver
import json

desired_cap = {
        "browser": "chrome",
        "browser_version": "latest",
        "os": "windows",
        "os_version": "10",
        'build': 'Python Accessibility Test Build',
        'name': 'Axe-core testing microsoft'
        }
driver = webdriver.Remote(
    desired_capabilities=desired_cap,
    command_executor='https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub'
    )
driver.get("https://www.microsoft.com")
axe_script = open('./axe.min.js', 'r')
driver.execute_script(axe_script.read())
axe_script.close()
result = driver.execute_async_script('var callback = arguments[arguments.length - 1];axe.run().then(results => callback(results))')
file = open("./report.json", "w")
file.write(json.dumps(eval(str(result))))
file.close()
driver.quit()

require	'rubygems'
require	'selenium-webdriver'
require 'json'

#Input Capabilities
caps = Selenium::WebDriver::Remote::Capabilities.new
caps["os"] = "Windows"
caps["os_version"] = "10"
caps["browser"] = "chrome"
caps["browser_version"] = "latest"
caps["build"] = "Sample Accessibility Test Build"
caps["name"] = "Accessibility test - Google"
caps["javascriptEnabled"] = "true"

driver = Selenium::WebDriver.for(:remote,
	:url => "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
	:desired_capabilities => caps)

	# Searching for 'BrowserStack' on google.com
	driver.navigate.to "http://www.google.com"
	axe_script = open('path/to/axe.min.js', 'r')
	driver.execute_script(axe_script.read())
	axe_script.close()
	result = driver.execute_async_script('var callback = arguments[arguments.length - 1];axe.run().then(results => callback(results))')
	file = open("path/to/report.json", "w")
	file.write(result.to_json)
	file.close()
	driver.quit

You can also conduct Accessibility testing of your websites on browsers where High Contrast Mode is enabled by setting BrowserStack’s custom capability browserstack.high_contrast to true. Check out high contrast mode to learn more.

Understand components of the JSON report

After you successfully run test script, a report.json file is generated as per the path set in the script. The report.json JSON file contains the following accessibility test information:

{
		testEngine: {…}
		passes: […]
		inapplicable: []
		url: “<URL of the website under test>”
		timestamp: “time when the json result is generated”
		testRunner: {…}
		toolOptions: {…}
		testEnvironment: {…}
		violations: {…}
		incomplete: {…}
}

Some of the information components available in the generated JSON report are:

violations (json): These results indicate what elements failed in the rules.
passes (array): These results indicate what elements passed in the rules.
incomplete (json): It contains results that were aborted and require further testing. This can happen either because of technical restrictions to what the rule can test or because a javascript error that occurred.
inapplicable (array): These results indicate rules that did not run because no matching content was found on the page. For example, if no video exists, those rules won’t run.

Each object returned in these arrays has some properties that can be found in the axe-core API documentation

Did this page help you?

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback

Is this page helping you?

Yes

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback!