Getting started with TestCafe

A guide to running TestCafe tests on BrowserStack

TestCafe is an end-to-end, pure Node.js solution for web app testing. It takes care of all the stages: starting browsers, running tests, gathering test results and generating reports. To execute TestCafe tests on BrowserStack you have to use the testcafe-browser-provider-browserstack plugin of TestCafe which integrates TestCafe with the BrowserStack testing cloud.

Installation

Assuming you already have testcafe installed. Use the following command to install the plugin:

npm i -g testcafe-browser-provider-browserstack

Before using this plugin, set up the BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY environment variables. You can also set up these variables with the following commands:

export BROWSERSTACK_USERNAME="YOUR_USERNAME"

export BROWSERSTACK_ACCESS_KEY="YOUR_ACCESS_KEY"

TestCafe needs these details to run your tests on BrowserStack. Visit account settings to find your username and access key under the Automate section.

You can now start running your TestCafe test suite on BrowserStack with the plug-in enabled and your BrowserStack credentials set up. To check that everything is working properly, execute the following command to get a list of BrowserStack environments (browsers and operating systems) available for your testing:

testcafe -b browserstack

The terminal will display a list of over 2000 available environments, depending on your BrowserStack account, if you’ve set up everything correctly.

"browserstack:chrome@79.0:Windows 10"
"browserstack:chrome@78.0:Windows 10"
"browserstack:chrome@77.0:Windows 10"
"browserstack:chrome@76.0:Windows 10"
...

The list indicates an available operating system and browser version. For example, "browserstack:Chrome@79.0:Windows 10" is an instance of version 79 of the Google Chrome browser running on Microsoft Windows 10.

Set up the BrowserStackLocal connection

TestCafe requires a BrowserStackLocal tunnel connection between your machine and the BrowserStack cloud to be able to execute your tests using the BrowserStack cloud. The tunnel connection can be set up in two ways:

User managed tunnel connection

1. Download the appropriate BrowserStackLocal binary applicable for your system. The following links will download the binary:
   • OSX
   • Linux-32 bit
   • Linux-64 bit
   • Windows
2. Start the binary using the command shown below (See the complete list of supported local arguments):

     ./BrowserStackLocal --key YOUR_ACCESS_KEY --local-identifier TestCafe --daemon start --parallel-runs <number-of-parallels>
   

   You can set <number-of-parallels> as the total number of parallels that your plan supports

3. Set the environment variable BROWSERSTACK_LOCAL_IDENTIFIER as shown below:

     export BROWSERSTACK_LOCAL_IDENTIFIER="TestCafe"
   

4. Now you can write your testcafe script or if you have a script ready then you can directly execute your tests on BrowserStack

5. After your tests are completed, stop the binary using the command as shown below:

     ./BrowserStackLocal --key YOUR_ACCESS_KEY --local-identifier TestCafe --daemon stop
   

TestCafe managed tunnel connection

Follow the below mentioned steps and TestCafe would set up the tunnel connection:

1. Download the appropriate BrowserStackLocal binary applicable for your system. The download links are available in the above section.

2. Once you have downloaded the binary, set the environment variable BROWSERSTACK_BINARY_PATH to the absolute path where the binary is located, as shown below. It will be used by TestCafe to start the BrowserStackLocal binary and create the tunnel connection that is required for running tests on BrowserStack.

     export BROWSERSTACK_BINARY_PATH="/Users/TestCafe/Desktop/BrowserStackLocal"
   

3. Setting the environment variable BROWSERSTACK_BINARY_PATH ensures that testcafe does not download the BrowserStackLocal binary every time it is invoked.

4. Additionally, you can set other environment variables to be provided as parameters for starting the BrowserStackLocal binary for handling advanced use-cases.

Writing a sample TestCafe test

The following testcafe test script example has been taken from DevExpress’s website. You can use it as a first test to run on BrowserStack:

// Save this file as test.js

// Save this file as test.js

import { Selector } from 'testcafe';

fixture('TestCafe example page').page(
  'https://devexpress.github.io/testcafe/example/'
);

test("Should submit developer's name successfully", async user => {
  // To interact with a DOM element, we must use the Selector function
  const interfaceSelect = Selector('#preferred-interface');
  const interfaceOption = interfaceSelect.find('option');

  await user
    .typeText('#developer-name', 'Sourav')
    .click('#remote-testing')
    .click(interfaceSelect)
    .click(interfaceOption.withText('Both'))
    .expect(interfaceSelect.value)
    .eql('Both')
    .click('#submit-button')
    .expect(Selector('#article-header').innerText)
    .eql('Thank you, Sourav!');
});

Executing tests on BrowserStack

BrowserStack supports two APIs for browser testing:

• BrowserStack JS Testing
• BrowserStack Automate

TestCafe uses the JS testing API by default. However, we recommend you to use BrowserStack Automate as it allows for much longer tests (2 hours) and additional features like window resizing, console logs and network logs by setting appropriate capabilities through environment variables or a config file (explained below).

To use Automate API, set BROWSERSTACK_USE_AUTOMATE environment variable to 1 as shown below:

export BROWSERSTACK_USE_AUTOMATE="1"

Running test scripts on BrowserStack

The syntax for running a TestCafe test script on BrowserStack is as shown below:

testcafe "browserstack:chrome@79.0:Windows 10" test.js

This invokes an instance of the specified environment on BrowserStack, runs all your tests automatically and then collects all the test results and generates a test report on your terminal as shown below:

TestCafe prints out the URL of the build returned by BrowserStack as shown in the test report. This URL takes you to your BrowserStack dashboard, where you can see the status of the build. You can also visit the Automate Dashboard and see the results of all your builds.

In the example shown above, all your test functions will execute as a part of a single BrowserStack session. If you want to speed up your testing and run multiple test functions at the same time, run your tests in parallel.

Using additional BrowserStack capabilities in tests

BrowserStack provides you a list of additional capabilities for your tests (view the complete list of capabilities). To use them, you have to set values in the respective environment variables as shown in the table below.

Run test scripts in incognito mode on Google Chrome

To execute a TestCafe test script in incognito mode on Google Chrome, follow these steps:

1. Set the BROWSERSTACK_USE_AUTOMATE environment variable to 1 by running the command:

     export BROWSERSTACK_USE_AUTOMATE="1"
   

2. Pass the --incognito argument to the BROWSERSTACK_CHROME_ARGS environment variable using the command:

     export BROWSERSTACK_CHROME_ARGS="--incognito"
   

3. Run the TestCafe test on a Google Chrome browser.

Use test scheduling functionality in tests

While executing a test having multiple test functions, the problem with TestCafe is that it executes all the test functions in the same browser session. Hence it becomes difficult to discover where a function was executed in a session. With concurrency >1 (see running tests on parallel), the problem becomes more severe because testcafe randomly runs different test functions in different/same browser sessions. Additionally, the name of the test is also unrecognizable by the user in the Automate dashboard.

All these issues could easily be resolved if you run your tests using Test Scheduling.

Test scheduling executes a test function as a separate test session on BrowserStack and helps in debugging the exact test function. The name of the test session is the same as the name of the test functions and hence it provides better identification for debugging purposes.

Prerequisites:

1. To enable test scheduling, install TestCafe to your local project directory using BrowserStack fork:

     npm i --save-dev @browserstack/testcafe
   

2. Install the testcafe-browser-provider-browserstack plugin using the BrowserStack fork as shown below:

     npm i --save-dev @browserstack/testcafe-browser-provider-browserstack
   

Executing test using test scheduling

The syntax for executing tests using --test-scheduling flag:

./node_modules/.bin/testcafe "<environment>" test.js --test-scheduling

Following is the example of executing TestCafe test on chrome 84 using --test-scheduling:

./node_modules/.bin/testcafe "@browserStack/browserstack:chrome@79.0:Windows 10" test.js --test-scheduling

The preceding command executes each of the test function of test.js as a separate session on BrowserStack specified environment.

You can also run your tests in parallel using --test-scheduling. For more information, see Running tests in parallel

Summary

By following the steps outlined in this guide and the guide for running your tests in parallel, you should be able to run your TestCafe tests on the 3000+ browsers and devices that are offered by BrowserStack’s infrastructure cloud at scale and execute your builds in a jiffy.