How to perform Visual Regression Testing using Cypress

Software Testing is the critical phase of the SDLC process where early testing saves a lot of cost and effort. A low-quality software can lower customer retention by impacting user engagement negatively. It is hence essential to build software that is bug-free to increase sales, making the product much more popular. There are different types of testing such as Smoke, Sanity, Integration, Regression, Unit Testing, and API Testing. Regression Testing is one of the important parts of Testing, especially when new features or updates are added to an existing application.

What is Regression Testing?

Regression Testing is the type of software testing process to ensure that the code change has adversely affected existing features. All or part of impacted features is tested as part of the regression testing process. Any issues which come into existence due to the latest code changes should be caught in this type of testing.

Role of Automation in Regression Testing

Unlike Sanity and Smoke, the scope of regression testing is vast, since the QA has to test the entire application or subset of features in regression testing. Considering that Manual Testing for Regression involves a lot of time and effort, also it is not cost-effective. This is the reason, why most organizations are opting for automated regression testing. Regression Testing in Agile Teams often works in Risk-based approach, opting for test automation frameworks for a streamlined test process.

Types of Regression Testing

Regression Testing can be classified into Functional Regression Testing and Visual Regression Testing. Fundamentally both work differently as explained below.

Functional Regression Testing

Function Regression Testing is the type of testing where workflow or use-cases will be tested using manual or automation tools. It involves entering some values by fetching selectors from browser DOM. This also includes validating text-based values by getting them either programmatically or manually. Most of the modern testing in practice is based on this approach.

Visual Regression Testing

A visual regression test checks what the user will see after any code changes have been executed. This is done by comparing screenshots taken before and after the code changes. This is why visual regression tests are also sometimes called visual snapshot tests. Visual test highlights any visual changes that occur after the code update.

Visual tests generate, analyze, and compare browser snapshots to detect if any pixels have changed. These pixel differences are called visual diffs (sometimes called perceptual diffs, pdiffs, CSS diffs, UI diffs). You can set the threshold values, which give you the flexibility to compare images with small differences or ignore small differences.

Visual Regression Testing using Cypress

Cypress is a popular test automation framework, it allows a lot of extensibility. Visual Regression Testing is one of the Extensible features of Cypress. There are a lot of plugins available in Cypress that can be used to capture visual images and compare them. This tutorial, explains Cypress Visual Testing using Cypress Image Diff plugin.

Step by Step Tutorial of Cypress Visual Regression

To set up Cypress Visual Regression environment, you need to install the pre-requisites listed below.

• Install NodeJS
• Install Visual Studio Code
• Install Cypress

Step 1: Install the cypress image diff npm package

Navigate your project Root folder (the directory where package.json is located). Enter the below command

npm i -D cypress-image-diff-js

Step 2: Configure Image Diff Plugin

Navigate to cypress/plugin/index.js and enter the commands written below

// cypress/plugin/index.js
module.exports = (on, config) => {
const getCompareSnapshotsPlugin = require('cypress-image-diff-js/dist/plugin')
getCompareSnapshotsPlugin(on, config)
}

Step 3: Import and add the Cypress image command

Navigate to cypress/support/commands.js and enter the code written below

// cypress/support/commands.js
const compareSnapshotCommand = require('cypress-image-diff-js/dist/command')
compareSnapshotCommand()

Step 4: Configure Reporter

Navigate to cypress/support/index.js and enter the code written below

after(() => {
cy.task('generateReport')
})

Step 5: Write your first Cypress Visual Test

Navigate to cypress/integration folder add new file example visual-test.js. Enter the code snippet written below

// visual-test.js
describe('Cypress Visual Testing', () => {
it('Compare fullpage of Google page', () => {
cy.visit("https://www.google.com/?hl=hi");
cy.compareSnapshot('google-page');
})
})

The above code, navigates to the Google Home page and compares if it is visually fine.

Step 6: Run your first Visual Regression Test with Cypress

Run your Cypress Visual Test using the below command.

npx cypress run --spec "cypress/integration/visual-test.js"

Step 7: View Report

One of the key features of this plugin is that it generates the good HTML report. Once you run the test, two folders will be created i.e.:

• cypress-visual-report: It contains, an HTML report, if the test passes there will not be any images shown if the test fails it will show baseline, comparison, and difference in that image.
• cypress-visual-screenshots: This folder contains three subfolders namely baseline, comparison, and diff, where each folder contains the respective image files.

Passed Test Report looks like below:

Failed Test Report Looks like the below:

How to perform Cypress Visual Testing of an element

Cypress can be used to perform visual tests on a specific element, where it compares the before and after screenshots of the specified elements as seen in the example below.

describe('Visuals', () => {
it('should compare screenshot from a given element', () => {
cy.visit('www.google.com')
cy.get('#report-header').compareSnapshot('search-bar-element')
})
})

This code snippet takes a snapshot of element #report-header and compares visually if it is same or not.

Adding Threshold to Visual Test

By default, the value of the threshold will be zero, which means it should match exactly with the base image. However, you can make it flexible by applying other threshold values.

Check out other useful functions of cypress-image-diff plugin

Executing Cypress Visual Comparision Test using BrowserStack

To run Cypress Visual Comparison Tests on Real Device Cloud like BrowserStack follow the steps mentioned below.

Step 1: Install BrowserStack Cypress plugin
Step 2: Create browserstack.json file using browserstack-cypress init command
Step 3: Copy and paste the below code

{
"auth": {
"username": "<my_username>",
"access_key": "<my_access_key>"
},
"browsers": [
{
"browser": "chrome",
"os": "Windows 10",
"versions": [
"latest",
"latest - 1"
]
}
],
"run_settings": {
"cypress_config_file": "./cypress.json",
"cypress_version": "9",
"project_name": "My sandbox project",
"build_name": "Build no. 1",
"parallels": "2",
"npm_dependencies": {
"cypress-image-diff-js": "^1.18.0"
}
}
}

Note:

• You can find the username and access key by logging into the browserstack website
• You can also change the browser settings and platform settings from browserstack.json file

Step 4: Configure cypress.json file to include the .js files.

{
"testFiles":["*.js"]
}

Step 5: Execute your Browserstack Test

Use the below command to execute Cypress Visual Test in Browserstack

browserstack-cypress run –sync

Run Cypress Tests on Real Device Cloud

Executing Cypress Visual Regression Test with Percy

Percy is a visual testing tool that helps in visual testing of an application. Since Percy is now a part of BrowserStack, you can access Percy Dashboard with BrowserStack credentials. You can use Percy with Cypress to perform a visual test by following the steps written below:

Step 1: Install Percy using the following command

npm install --save-dev @percy/cli @percy/cypress

Step 2: To import Percy to Cypress, navigate to cypress/support/index.js File and enter the following command

import '@percy/cypress'

Step 3: Write your first Percy Visual Test Script as shown below

describe('Cypress Visual Testing', () => {
it('Compare fullpage of Google page', () => {
cy.visit("https://www.google.com/?hl=hi");
cy.percySnapshot('Google');
})
})

This example does a visual comparison of the Google.com Page using Percy and Cypress

Run your Percy Test

Go to http://percy.io create a sample project. Once you create a Sample Project Percy will generate API key as seen below. Copy the Percy Token from here.

Enter the Percy Token using the following command for MacOS

export PERCY_TOKEN=<your_token>

For Windows OS, enter the Percy Token using the command written below

set PERCY_TOKEN=<your_token>

For powershell use below command to enter Percy Token

$env:PERCY_TOKEN ==<your_token>

Once the Percy Token is entered use below command to run your Percy Cypress Tests

npx percy exec -- cypress run

Cypress Visual Tests Starts Running and it gives you following result in command line