How to Perform Visual Regression Testing Using Playwright

Regression Testing verifies that system changes do not interfere with existing features or code structure. They are part of almost every test suite in software development life cycles. It is common for developers to change or add a code section and unintentionally disrupt something that is working just fine.

Visual Regression Testing functions on the same logic but confines it to the visual aspects of the software. It works by comparing two images and automating complicated scenarios, like when we cannot identify the elements in the DOM tree. However, visual regression can be used on any website.

How does Visual Regression Testing in Playwright work?

During the first run, the visual comparison tool captures the snapshot called the base image. The subsequent run compares the base image if there is no difference test is passed, and if there is a difference, the test is considered as failed. Visual Regression is also called Visual Comparison Testing.

In this tutorial, we will discuss Automated Visual Regression Using Playwright.

Pre-requisites for Visual Regression with Playwright:

• Download and install NodeJS
• Download and install Visual Studio Code (Recommended)
• Install Playwright NPM module
• Install @playwright/test module

Playwright comes with the default Visual Comparison Tool, so there is no need to install additional packages.

Create Simple Visual Tests using Playwright

In your tests folder, create a new javascript file example demo.spec.jspage.screenshot() function takes the screenshot, and expect in the @playwright/test module provides the assertion for matching the images that are .toMatchSnapshot().

Inside the newly created javascript file, create a new test that performs the visual comparison like below.

// example.spec.js

const { test, expect } = require('@playwright/test');

test('Visual Comparison Test Demo', async ({ page }) => {

  await page.goto('https://playwright.dev');

  expect(await page.screenshot()).toMatchSnapshot();

});

In the above code, we navigate to the official Playwright website and compare visually by taking a snapshot.

Run your First Visual Regression Tests

Run your visual comparison tests using the below command

npx playwright test

The first time the test is run, it fails, as there is no base image.

As seen below, the directory containing the base image file gets created inside the tests folder.

Run the Spec again; it passes

Visual Snapshot Comparison in Playwright Ignore Minor Differences

The above comparison technique matches the screenshot pixel by pixel, which means each pixel should match exactly. This behavior can be modified by passing the argument maxDiffPixels = <pixel_value>.

Example:

const { test, expect } = require('@playwright/test');

test('Visual Comparison Test Demo', async ({ page }) => {

  await page.goto('https://playwright.dev');

  expect(await page.screenshot()).toMatchSnapshot({ maxDiffPixels: 200 });

});

In the above example

We have specified the maxDiffPixels value as 200, which means the maximum pixel difference can be  200.

Image Comparison in Playwright with Threshold option

Playwright toMatchSnapshot() accepts threshold, threshold ranges from 0 to 1, default value is 0.2. The threshold is tolerance of image differences.

Example Code:

const { test, expect } = require('@playwright/test');

test('Visual Comparison Test Demo', async ({ page }) => {

  await page.goto('https://playwright.dev/');

  expect(await page.screenshot()).toMatchSnapshot({threshold:0.5});

});

In the above code, the threshold is mentioned as 0.5.

Playwright Visual Comparison Tips & Tricks 

• In Playwright, we can pass the image file name; instead of default comparison, Playwright compares with the specified filename.

Example:

 expect(await page.screenshot()).toMatchSnapshot('home.png');

• Playwright also allows us to compare element snapshots; we can take a snapshot of DOM elements and compare.

Example: 

expect(await page.locator('xpath=//*[@id="__docusaurus']).screenshot()).toMatchSnapshot();

Visual Testing with Playwright using Percy

Percy is a Web-based tool for Visual Testing, and it provides both manual and automation capability for Visual Comparison. Percy supports Playwright integration. 

Percy is now a part of Browserstack, if you already have a BrowserStack account, you can sign in with BrowserStack or sign up and create one.

Take Snapshots in Playwright using Percy 

Step 1 – Install Percy modules using the following command.

npm install --save-dev @percy/cli @percy/playwright

Step 2 – Create a new Javascript Playwright Test file like below.

//demo.spec.ts

const { chromium } = require('playwright');

const percySnapshot = require('@percy/playwright');

(async () => {

  const browser = await chromium.launch();

  const page = await browser.newPage();

  await page.goto('https://www.browserstack.com/', { waitUntil: 'networkidle' });

  await percySnapshot(page, 'Example Site');

  await browser.close();

})();

In the above example, we are navigating to https://www.browserstack.com/, and we are taking a snapshot using the percySnapshot() function.

Setting Up Percy

Step 1 – Login to Percy. If you don’t have an account create one

Step 2 – Create a new project 

Step 3 – Copy Percy Token

Step 4 – In your Visual Studio Code Terminal Set the PERCY_TOKEN environment variable using the below commands

Powershell / Visual Studio Code Terminal

$env:PERCY_TOKEN = "your_token"

Windows Command Line

set PERCY_TOKEN="your_token"

Unix/Linux Based OS

export PERCY_TOKEN="your_token"

Step 5 – Run your tests using the below command

npx percy exec -- node tests\demo.spec.js

Step 6 – Start Execution and wait until it finishes

Once Percy completes tests it shows in the console log.

Console Log provides the build URL. Once you navigate to that URL, you can see your image uploaded into the Percy.

If there is no difference in the image it says “No Changes”, like below.

If there is an image difference, it will show the difference side by side like below.

Try Visual Regression Testing for Free