How to Perform Visual Regression Puppeteer

Visual aesthetics are essential to engage customers and play a major role in customer retention. It is important to ensure that the website is visually consistent even after new deployments and should be tested before every release.

Visual Testing is a user interface testing technique where the base images (before deployment) are matched with actual images (after deployment) using pixel-by-pixel comparison. Based on the difference between the base and actual image, test will be marked as pass or fail. Visual Testing is also called Visual Validation Testing or Visual Regression Testing.

Visual Regression Tests can be automated, just like Functional Regression Tests. Visual Test Automation can be performed using different frameworks such as Cypress, Playwright, Selenium, TestCafe, Storybook, WebdriverIO, etc.  

This article discusses Visual Testing with Puppeteer. 

What is Puppeteer?

Puppeteer is the automation testing framework managed by the Google team. It is built with NodeJS and allows us to write code in JavaScript or TypeScript. Puppeteer, by default, doesn’t have any assertion libraries, so you need to integrate assertion libraries such as Mocha, Jasmine, or Jest.

Pre-Requisite for Puppeter Visual Regression Testing

You need to Set up Basic Puppeteer Framework to perform Visual Regression Tests with Puppeteer.

There are several ways of performing Visual Regression in Puppeteer. This article highlights the following two methods:

1. Puppeteer Visual Validation Test using jest-image-snapshot 
2. Puppeteer Visual Testing using Percy

Puppeteer Visual Validation Test using jest-image-snapshot

The jest-image-snapshot is a node package it provides integration with Jest and Puppeteer with minimal settings. 

Follow the below steps  to Perform Visual Testing using Puppeteer and Jest:

Step 1: Install jest-image-snapshot package using the command

npm i --save-dev jest-image-snapshot

Step 2: Add the visual regression test

Let’s take the snapshot after navigating to https://www.browserstack.com home page and compare them using the code below.

//visual.test.js

const { toMatchImageSnapshot } = require('jest-image-snapshot');

expect.extend({ toMatchImageSnapshot });

describe('Visual Testing', () => {

    jest.setTimeout(50000);

    beforeAll(async () => {

           await page.goto('https://www.browserstack.com')

    })


    it('Visual Regression Test', async () => {

        const image = await page.screenshot();


        expect(image).toMatchImageSnapshot({

            failureThreshold: '0.10',

            failureThresholdType: 'percent'

        });

    })

})

In the above code

const { toMatchImageSnapshot } = require('jest-image-snapshot');

Importing the toMatchImageSnapshot constant, which helps to match the images.

expect.extend({ toMatchImageSnapshot });

By default Jest’s expect doesn’t have an image comparison technique, so you need to extend the functionality with expect.extend();

const image = await page.screenshot(); 

The screenshot will be taken and saved in the variable image

expect(image).toMatchImageSnapshot({
            failureThreshold: '0.10',
            failureThresholdType: 'percent'
        });

The above code compares the screenshot. failureThreshold makes image comparison more flexible by adding image difference tolerance. 

Step 3: Execute your Puppeteer Visual Tests using Jest by entering the below command

npx jest -c ./jest.config.js

Alternatively, if you have configured the tests command in package.json you can simply execute it using the below command.

npm run test

The first time when you execute the test, the Puppeteer captures the base screenshot. In subsequent runs, it compares the screenshot with the base screenshot.

After the first execution, the __image_snapshots__folder will be created, and it saves the base snapshot file inside the folder.

The pass/fail test results will be shown on the command line, as seen below.

If there is a difference in base and actual screenshots, then the test will be marked as a fail.

If you navigate to diff-output folder, the image will be shown in side by side comparison.

Puppeteer Visual Testing for DOM Element

Following the above method, you will be able to perform visual testing for the complete page. However, there might be certain scenarios in which you need to perform visual validation for Single DOM elements such as Button, Checkbox, Menu, etc. Puppeteer supports Visual Test for Single DOM Element. You just need to change the code to run Visual Tests for DOM Element.

Instead of taking the screenshot at the page level, you can take the screenshot at the element level, like below.

 const element = await page.$('#signupModalButton');

 const image = await element.screenshot();

In the above code, the element is captured first, and then the screenshot is taken.

The complete code of Puppeteer Visual Testing for DOM Element, looks like the snippet below.

//visual.test.js

const { toMatchImageSnapshot } = require('jest-image-snapshot');

expect.extend({ toMatchImageSnapshot });

describe('Visual Testing', () => {

    jest.setTimeout(50000);

    beforeAll(async () => {

        expect.extend(toMatchImageSnapshot);

        await page.goto('https://www.browserstack.com')

    })


    it('Visual Regression Test', async () => {

        const element = await page.$('#signupModalButton');

        const image = await element.screenshot();


        expect(image).toMatchImageSnapshot({

            failureThreshold: '0.10',

            failureThresholdType: 'percent'

        });

    })

})

The execution step remains the same as that used in the previous section.

Puppeteer Visual Testing Using Percy

Percy is cloud-based Visual Comparison tool. It supports various test automation frameworks such as Storybook, Selenium, Cypress, Playwright, TestCafe, and WebdriverIO.

Percy can be integrated with Puppeteer too. As Percy provides the dedicated dashboard for visual comparison, this helps in reviewing and approving visual tests. Each time when the Percy test has been executed, the build will be triggered, so it is easy to maintain the historical results.

Also, root cause analysis and debugging of Visual Tests becomes easier with Percy as it allows you to share test reports over Slack.

Try Percy for Free

Integrating Percy for Puppeteer Jest 

Step 1: Install @percy/puppeteer and @percy/cli using npm:

npm install --save-dev @percy/cli @percy/puppeteer

Step 2: Write Puppeteer Tests using Percy

In the Puppeteer test, the percySnapshot needs to be imported in order to take screenshots using the below command

const percySnapshot = require('@percy/puppeteer')

Once you import the percySnapshot, you can perform a set of actions and then take a screenshot at the required step.

For example, navigating to browserstack.com homepage first, then taking a screenshot and comparing for visual testing, the scenario can be written as:

//visual-percy.test.js

const percySnapshot = require('@percy/puppeteer')

describe('Visual Testing', () => {

    jest.setTimeout(50000);

    beforeAll(async () => {

        await page.goto('https://www.browserstack.com')

    })

    it('Visual Regression Test', async () => {

        await percySnapshot(page, "percy-visual-test-puppeteer-jest")

    })

})

Step 3: Create Percy Project using the below steps

• Login to Percy
• Create a new Project

Step 4: Set the PERCY_TOKEN environment variable

Once a new Project is created, Navigate to Settings and Copy the Percy Token.

To run the Percy test, you need to set the environment variable. The setting up environment variable depends on the operating system and command-line tool you are using. Here’s how you can set PERCY_TOKEN for different CLIs, such as:

• Windows Command Line

set PERCY_TOKEN=<your token here>

• Mac/Linux Terminal

export PERCY_TOKEN=<your token here>

• Powershell Terminal

$env: PERCY_TOKEN="<your token here>"

Once the environment variable is set up, you are ready to execute your test.

Step 5: Execute your test using the command.

Execute Percy Visual Test for Puppeteer and Jest using the below command

npx percy exec -- jest -c ./jest.config.js

Wait until the execution finishes. Once execution is complete, you will see the build URL in the command line.

Navigate to the build URL, and you will be landed in Percy Build Dashboard. If there is any difference in the images, the Percy dashboard shows the diff in side by side, as seen below.

If there is no difference, then Percy’s dashboard shows the image with No Changes, as seen in the following screenshot.

Conclusion

Puppeteer was originally designed to perform functional tests. However, it can also be used for Visual Validation. Remember, visual testing is recommended only for validation of web application’s user interface changes. The functionality and dynamic data cannot be verified using Visual Regression Tests. 

Visual regression test makes UI testing easy and quick. It helps deliver consistent visuals and hence a seamless user experience. It is usually recommended for projects with frequent releases, and visual testing can be a barrier in Continuous Deployment.