Getting started with TestCafe

A guide to running TestCafe tests on BrowserStack

TestCafe is an end-to-end, pure Node.js solution for web app testing. It takes care of all the stages: starting browsers, running tests, gathering test results and generating reports. To execute TestCafe tests on BrowserStack you have to use the testcafe-browser-provider-browserstack plugin of TestCafe which integrates TestCafe with the BrowserStack testing cloud.

Installation

Assuming you already have testcafe installed. Use the following command to install the plugin:

npm i -g testcafe-browser-provider-browserstack

Before using this plugin, set up the BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY environment variables. You can also set up these variables with the following commands:

export BROWSERSTACK_USERNAME="YOUR_USERNAME"
export BROWSERSTACK_ACCESS_KEY="YOUR_ACCESS_KEY"

TestCafe needs these details to run your tests on BrowserStack. Click here to find your username and access key under the Automate section.

You can now start running your TestCafe test suite on BrowserStack with the plug-in enabled and your BrowserStack credentials set up. To check that everything is working properly, execute the following command to get a list of BrowserStack environments (browsers and operating systems) available for your testing:

testcafe -b browserstack

The terminal will display a list of over 2000 available environments, depending on your BrowserStack account, if you’ve set up everything correctly.

"browserstack:chrome@79.0:Windows 10"
"browserstack:chrome@78.0:Windows 10"
"browserstack:chrome@77.0:Windows 10"
"browserstack:chrome@76.0:Windows 10"
...

The list indicates an available operating system and browser version. For example, "browserstack:Chrome@79.0:Windows 10" is an instance of version 79 of the Google Chrome browser running on Microsoft Windows 10.

Set up the BrowserStackLocal connection

TestCafe requires a BrowserStackLocal tunnel connection between your machine and the BrowserStack cloud to be able to execute your tests using the BrowserStack cloud. The tunnel connection can be set up in two ways:

User managed tunnel connection

Protip:
  • The stability of tests run on BrowserStack is much better if the BrowserStackLocal binary is managed by the user and not TestCafe.
  • Follow the steps in this section to start and stop the binary and set the BROWSERSTACK_LOCAL_IDENTIFIER variable with the identifier for your tunnel so that TestCafe does not create another tunnel connection.
  1. Download the appropriate BrowserStackLocal binary applicable for your system. The following links will download the binary:
  2. Start the binary using the command shown below (See the complete list of supported local arguments):
      ./BrowserStackLocal --key YOUR_ACCESS_KEY --local-identifier TestCafe --daemon start --parallel-runs <number-of-parallels>
    

    You can set <number-of-parallels> as the total number of parallels that your plan supports

  3. Set the environment variable BROWSERSTACK_LOCAL_IDENTIFIER as shown below:
      export BROWSERSTACK_LOCAL_IDENTIFIER="TestCafe"
    
  4. Now you can write your testcafe script or if you have a script ready then you can directly execute your tests on BrowserStack

  5. After your tests are completed, stop the binary using the command as shown below:
      ./BrowserStackLocal --key YOUR_ACCESS_KEY --local-identifier TestCafe --daemon stop
    

TestCafe managed tunnel connection

Warning:
  • This is a comparatively hassle free way of managing the BrowserStackLocal tunnel connection
  • Stability of tests might be negatively impacted if this option is chosen

Follow the below mentioned steps and TestCafe would set up the tunnel connection:

  1. Download the appropriate BrowserStackLocal binary applicable for your system. The download links are available in the above section.

  2. Once you have downloaded the binary, set the environment variable BROWSERSTACK_BINARY_PATH to the absolute path where the binary is located, as shown below. It will be used by TestCafe to start the BrowserStackLocal binary and create the tunnel connection that is required for running tests on BrowserStack.
      export BROWSERSTACK_BINARY_PATH="/Users/TestCafe/Desktop/BrowserStackLocal"
    
  3. Setting the environment variable BROWSERSTACK_BINARY_PATH ensures that testcafe does not download the BrowserStackLocal binary every time it is invoked.

  4. Additionally, you can set other environment variables to be provided as parameters for starting the BrowserStackLocal binary for handling advanced use-cases.

Writing a sample TestCafe test

The following testcafe test script example has been taken from DevExpress’s website. You can use it as a first test to run on BrowserStack:

// Save this file as test.js

import { Selector } from 'testcafe';

fixture('TestCafe example page').page(
  'https://devexpress.github.io/testcafe/example/'
);

test("Should submit developer's name successfully", async user => {
  // To interact with a DOM element, we must use the Selector function
  const interfaceSelect = Selector('#preferred-interface');
  const interfaceOption = interfaceSelect.find('option');

  await user
    .typeText('#developer-name', 'Sourav')
    .click('#remote-testing')
    .click(interfaceSelect)
    .click(interfaceOption.withText('Both'))
    .expect(interfaceSelect.value)
    .eql('Both')
    .click('#submit-button')
    .expect(Selector('#article-header').innerText)
    .eql('Thank you, Sourav!');
});

Executing tests on BrowserStack

BrowserStack supports two APIs for browser testing:

TestCafe uses the JS testing API by default. However, we recommend you to use BrowserStack Automate as it allows for much longer tests (2 hours) and additional features like window resizing, console logs and network logs by setting appropriate capabilities through environment variables or a config file (explained below).

To use Automate API, set BROWSERSTACK_USE_AUTOMATE environment variable to 1 as shown below:

export BROWSERSTACK_USE_AUTOMATE="1"
Note:
  • testcafe -b browserstack command gives a list of supported browsers for the BrowserStack JS Testing API. If you are using BrowserStack Automate (i.e. BROWSERSTACK_USE_AUTOMATE=”1" to run your Testcafe tests, then see the list of supported browsers and platforms by selecting the OS/browser/device combination
  • For testing on real mobile devices, the browser and versions are not required. Only mentioning the device name is sufficient. For e.g. “browserstack:iPhone 8” or "browserstack:Samsung Galaxy S20 Ultra”

Running test scripts on BrowserStack

The syntax for running a TestCafe test script on BrowserStack is as shown below:

testcafe "browserstack:chrome@79.0:Windows 10" test.js

This invokes an instance of the specified environment on BrowserStack, runs all your tests automatically and then collects all the test results and generates a test report on your terminal as shown below:

testcafe-test-execution-result.png

TestCafe prints out the URL of the build returned by BrowserStack as shown in the test report. This URL takes you to your BrowserStack dashboard, where you can see the status of the build. You can also visit the Automate Dashboard and see the results of all your builds.

In the example shown above, all your test functions will execute as a part of a single BrowserStack session. If you want to speed up your testing and run multiple test functions at the same time, run your tests in parallel.

Using additional BrowserStack capabilities in tests

BrowserStack provides you a list of additional capabilities for your tests (view the complete list of capabilities). To use them, you have to set values in the respective environment variables as shown in the table below.

Note: These capabilities are only supported only if you are using Automate API to run your tests i.e. BROWSERSTACK_USE_AUTOMATE variable is set to 1
Capabilities Description
BROWSERSTACK_PROJECT_NAME It allows you to name your build, which is useful for keeping track of test runs on the BrowserStack dashboard.
BROWSERSTACK_BUILD_ID It allows you to give a name to your build which helps you to keep track of all the test session in the Automate Dashboard
BROWSERSTACK_TEST_RUN_NAME This environment variable is useful only if you are running tests serially. For naming tests that you want to run in parallel, run tests using test scheduling
BROWSERSTACK_ACCEPT_SSL_CERTS Used to ignore invalid certificate error in your test.
Default value is false
BROWSERSTACK_DEBUG Required if you want to generate screenshots at various steps in your test.
Default value is false
BROWSERSTACK_CONSOLE Required if you want to capture browser console logs at various steps in your test. Console Logs are available for Selenium tests on Desktop Chrome, Desktop Firefox, Mobile Chrome (Android devices) and Mobile Safari (iOS devices).
Possible values: disable, errors, warnings, info, verbose
BROWSERSTACK_DISPLAY_RESOLUTION Sets the display resolution. A complete list of network resolutions can be found here.
Default value is 1024x768
BROWSERSTACK_GEO_LOCATION Required if you want to test how your tests behave in specific countries.
Possible values are CN, FR, IN, US
BROWSERSTACK_TIMEZONE Required if you want to configure tests to run on a custom time zone.
Possible values: New_York, Los_Angeles, La_Rioja. You can view the complete list of timezones on Wikipedia.
BROWSERSTACK_VIDEO Required if you want to enable video recording during your test.
Default value is true
BROWSERSTACK_NETWORK_LOG Required if you want to capture network logs for your test. Network logs are supported for all desktop browsers, Android, and iOS devices with a few exceptions - IE 10 on any OS; IE 11 on Windows 7 / 8.1, and any browser on macOS High Sierra and Mojave.
Protip:
  • Any additional capabilities supported by BrowserStack can be used in TestCafe tests by saving that in the file browserstack-config.json and saving the path of the file in the environment variable as shown below:
    export BROWSERSTACK_CAPABILITIES_CONFIG_PATH="./Folder/browserstack-config.json"
  • The browserstack-config.json file should be structured like shown below:
      {
         "browserstack.idleTimeout":    "300"
    }
    
  • Any capability that you set in the browserstack-config.json file, will override the same capability (if set), that has been set using environment variables
  • These capabilities are only applicable only if your are using Automate API for testing i.e. BROWSERSTACK_USE_AUTOMATE = "1"

Use test scheduling functionality in tests

While executing a test having multiple test functions, the problem with TestCafe is that it executes all the test functions in the same browser session. Hence it becomes difficult to discover where a function was executed in a session. With concurrency >1 (see running tests on parallel), the problem becomes more severe because testcafe randomly runs different test functions in different/same browser sessions. Additionally, the name of the test is also unrecognizable by the user in the Automate dashboard.

All these issues could easily be resolved if you run your tests using Test Scheduling.

Test scheduling executes a test function as a separate test session on BrowserStack and helps in debugging the exact test function. The name of the test session is the same as the name of the test functions and hence it provides better identification for debugging purposes.

Prerequisites:

  1. To enable test scheduling, install TestCafe to your local project directory using BrowserStack fork:
      npm i --save-dev @browserstack/testcafe
    
  2. Install the testcafe-browser-provider-browserstack plugin using the BrowserStack fork as shown below:
      npm i --save-dev @browserstack/testcafe-browser-provider-browserstack
    

Executing test using test scheduling

Note: Now that you have installed testcafe from the BrowserStack fork, make sure you run that locally installed testcafe version in the following steps.

The syntax for executing tests using --test-scheduling flag:

./node_modules/.bin/testcafe "<environment>" test.js --test-scheduling

Following is the example of executing TestCafe test on chrome 84 using --test-scheduling:

./node_modules/.bin/testcafe "@browserStack/browserstack:chrome@79.0:Windows 10" test.js --test-scheduling

The preceding command executes each of the test function of test.js as a separate session on BrowserStack specified environment.

You can also run your tests in parallel using --test-scheduling. For more information, see Running tests in parallel

Summary

By following the steps outlined in this guide and the guide for running your tests in parallel, you should be able to run your TestCafe tests on the 2000+ browsers and devices that are offered by BrowserStack’s infrastructure cloud at scale and execute your builds in a jiffy.

We're sorry to hear that. Please share your feedback so we can do better






Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked





Thank you for your valuable feedback