App & Browser Testing Made Easy

Give your users a seamless experience by testing on 3000+ real devices and browsers. Don't compromise with emulators and simulators

Home Guide How to Perform Visual Regression Testing Using Playwright

How to Perform Visual Regression Testing Using Playwright

By Ganesh Hegde, Community Contributor -

Table of Contents

Regression Testing verifies that system changes do not interfere with existing features or code structure. They are part of almost every test suite in software development life cycles. It is common for developers to change or add a code section and unintentionally disrupt something that is working just fine.

Visual Regression Testing functions on the same logic but confines it to the visual aspects of the software. It works by comparing two images and automating complicated scenarios, like when we cannot identify the elements in the DOM tree. However, visual regression can be used on any website.

How does Visual Regression Testing work?

During the first run, the visual comparison tool captures the snapshot called the base image. The subsequent run compares the base image if there is no difference test is passed, and if there is a difference, the test is considered as failed. Visual Regression is also called Visual Comparison Testing.

In this tutorial, we will discuss Automated Visual Regression Using Playwright.

Pre-requisites for Visual Regression with Playwright:

Note: Throughout this tutorial, we are using Playwright with Javascript.

Playwright comes with the default Visual Comparison Tool, so there is no need to install additional packages.

Create Simple Visual Comparison Tests using Playwright

In your tests folder, create a new javascript file example demo.spec.jspage.screenshot() function takes the screenshot, and expect in the @playwright/test module provides the assertion for matching the images that are .toMatchSnapshot().

Inside the newly created javascript file, create a new test that performs the visual comparison like below.

// example.spec.js

const { test, expect } = require('@playwright/test');

test('Visual Comparison Test Demo', async ({ page }) => {

  await page.goto('https://playwright.dev');

  expect(await page.screenshot()).toMatchSnapshot();

});

In the above code, we navigate to the official Playwright website and compare visually by taking a snapshot.

Run your First Visual Regression Tests

Run your visual comparison tests using the below command

npx playwright test

NPX Playwright

The first time the test is run, it fails, as there is no base image.

As seen below, the directory containing the base image file gets created inside the tests folder.

Demo Specs

Run the Spec again; it passes

Demo Specs Playwright for Visual Regression Testing

Visual Comparison in Playwright Ignore Minor Differences

The above comparison technique matches the screenshot pixel by pixel, which means each pixel should match exactly. This behavior can be modified by passing the argument maxDiffPixels = <pixel_value>.

Example:

const { test, expect } = require('@playwright/test');

test('Visual Comparison Test Demo', async ({ page }) => {

  await page.goto('https://playwright.dev');

  expect(await page.screenshot()).toMatchSnapshot({ maxDiffPixels: 200 });

});

In the above example

We have specified the maxDiffPixels value as 200, which means the maximum pixel difference can be  200.

Image Comparison in Playwright with Threshold option

Playwright toMatchSnapshot() accepts threshold, threshold ranges from 0 to 1, default value is 0.2. The threshold is tolerance of image differences.

Example Code:

const { test, expect } = require('@playwright/test');

test('Visual Comparison Test Demo', async ({ page }) => {

  await page.goto('https://playwright.dev/');

  expect(await page.screenshot()).toMatchSnapshot({threshold:0.5});

});

In the above code, the threshold is mentioned as 0.5.

Playwright Visual Comparison Tips & Tricks 

  • In Playwright, we can pass the image file name; instead of default comparison, Playwright compares with the specified filename.

Example:

 expect(await page.screenshot()).toMatchSnapshot('home.png');
  • Playwright also allows us to compare element snapshots; we can take a snapshot of DOM elements and compare.

Example: 

expect(await page.locator('xpath=//*[@id="__docusaurus']).screenshot()).toMatchSnapshot();

Visual Regression with Playwright using Percy

Percy is a Web-based tool for Visual Testing, and it provides both manual and automation capability for Visual Comparison. Percy supports Playwright integration

Percy is now a part of Browserstack, if you already have a BrowserStack account, you can sign in with BrowserStack or sign up and create one.

Using Percy with Playwright

Step 1 – Install Percy modules using the following command.

npm install --save-dev @percy/cli @percy/playwright

Step 2 – Create a new Javascript Playwright Test file like below.

//demo.spec.ts

const { chromium } = require('playwright');

const percySnapshot = require('@percy/playwright');

(async () => {

  const browser = await chromium.launch();

  const page = await browser.newPage();

  await page.goto('https://www.browserstack.com/', { waitUntil: 'networkidle' });

  await percySnapshot(page, 'Example Site');

  await browser.close();

})();

In the above example, we are navigating to https://www.browserstack.com/, and we are taking a snapshot using the percySnapshot() function.

Setting Up Percy

Step 1 – Login to Percy. If you don’t have an account create one

Step 2 – Create a new project 

Step 3 – Copy Percy Token

Step 4 – In your Visual Studio Code Terminal Set the PERCY_TOKEN environment variable using the below commands

Powershell / Visual Studio Code Terminal

$env:PERCY_TOKEN = "your_token"

Windows Command Line

set PERCY_TOKEN="your_token"

Unix/Linux Based OS

export PERCY_TOKEN="your_token"

Step 5 – Run your tests using the below command

npx percy exec -- node tests\demo.spec.js

Note: tests\demo.spec.js is our test file name, provide the relative path of your test file name.

Step 6 – Start Execution and wait until it finishes

Once Percy completes tests it shows in the console log.

Setting Up Percy command

Console Log provides the build URL. Once you navigate to that URL, you can see your image uploaded into the Percy.

If there is no difference in the image it says “No Changes”, like below.

Percy No Changes

If there is an image difference, it will show the difference side by side like below.

NPX Playwright

Try Visual Regression Testing for Free

Tags
Automated UI Testing Automation Testing UI Testing Visual Testing

Featured Articles

5 Core Practices for Efficient Visual Testing

How to Run Visual Tests with Selenium: Tutorial

How to perform Visual Regression Testing using Cypress