HomeGuideHow to perform Visual Regression Testing in NightwatchJS
How to perform Visual Regression Testing in NightwatchJS
By Ganesh Hegde, Community Contributor -October 3, 2022
Automation tools are primarily used to test web application functionality, but the modern web design trend has changed. The User Interface and User Experience (UI/UX) are getting the same importance as functionality and the best way to UI/UX elements is through Visual Regression Testing.
Visual Regression testing compares two screenshots, a baseline screenshot, and an actual screenshot. The baseline screenshot can be considered as an expected screenshot and the actual screenshot should always match the baseline screenshot. If it fails to match, the test can be marked as a fail.
NightwatchJS is a NodeJS-based open-source test automation framework that allows users to automate web applications with a bundle of features such as clear syntax, a built-in test runner, cloud testing support, page object support, and continuous integration. It supports the custom commands feature, where the user can define the custom commands and add the functionality for that command. Simply put, you can hook any plugins like node packages and use them in your automation scripts. Using this feature, we can automate the Visual Validation Testing in NightwatchJS.
This article will get into two different ways to perform Visual Regression Testing in NightwatchJS
//demoVRT.js
describe('Nightwatch Visual Test Demo', function () {
it('Nightwatch Visual Test', async function (browser) {
await browser
.navigateTo('https://www.browserstack.com/')
.assert.compareScreenshot('First test', 'Screenshot captured!')
.end();
});
});
Step 4: Execute the Test
Execute Nightwatch Visual Validation test using the below command\
Syntax:
npx nightwatch <path_to_test_file>
Example:
npx nightwatch tests/demoVRT.js
Note: The baseline screenshot is captured, when you execute the test the first time, so to get an actual comparison test needs to be executed at least 2 times.
Step 5: View the result
Once you execute the script the output will be shown in the command line terminal.
If there is no difference in the screenshots, then the test will be marked as Pass.
If there is any difference in the image, then the test fails
The folder tests_output/visual-regression contains multiple images, the actual image, the base image, and the difference between the actual and base image.
Visual Testing for Single HTML Element
The plugin also supports Visual Testing for a single HTML element with the command compareElementScreenshot().
//demoVRT.js
describe('Nightwatch Visual Test Demo', function () {
it('Nightwatch Visual Test', async function (browser) {
await browser
.navigateTo('https://www.browserstack.com/docs')
.assert.compareElementScreenshot('#product-menu-toggle','First test', 'Screenshot captured!')
.end();
});
});
Performing Nightwatch Visual Regression Testing with Percy
Percy is a popular visual testing tool that supports the visual testing of multiple browsers. It provides a dedicated build dashboard for each build, which helps the users view the test results online, and also provides functions such as Review, Accept and Reject the build. Percy supports both Manual and Automated visual testing.
Advantages of Percy
Percy’s visual testing reports can be viewed online.
One can perform a review online and take action such as accept, reject or request changes.
Percy has an inbuilt comparison mechanism for all browsers so rendered screenshots can be viewed for multiple browsers
Percy supports both component and Page based visual testing.
Percy allows you to adjust the diff sensitivity (same as screenshot threshold)
The above command installs the two packages @percy/cli and @percy/nightwatch. The @percy/nightwatch provides the custom commands for visual comparison and @percy/cli provides functionality to execute the test and upload to Percy infrastructure.
Step 2: Configure Percy for Nightwatch
Import the Percy package
Open the nightwatch.conf.js file and add the required statement
const percy = require('@percy/nightwatch');
Note: The above require statement should be before module.exports
Add the custom command
In your nightwatch.conf.js file search for custom_commands_path: and add the Percy command like below.
custom_commands_path: [percy.path],
Step 3: Create a simple Percy Nightwatch script
//demoVRT.js
describe('Nightwatch Visual Test Demo', function () {
it('Nightwatch Visual Test', async function (browser) {
await browser
.navigateTo('https://www.browserstack.com/docs')
.percySnapshot('Demo snapshot')
.end();
});
});
In the above code, we are navigating to the BrowserStack Documentation page and taking the screenshot for visual comparison.
The percySnapshot(‘Demo snapshot’) command saves the screenshot with the name ‘Demo snapshot’
Step 4: Copy the PERCY_TOKEN
Login to Percy (If you don’t have an account, create one by Signing Up)
Create a New Project
Navigate to Project Settings
Copy the PERCY_TOKEN
Step 5: Set the PERCY_TOKEN environment variable
Based on the command-line tool you are using, you can set the environment variable as below.
# Unix
$ export PERCY_TOKEN="<your-project-token>"
# Windows
$ set PERCY_TOKEN="<your-project-token>"
# Powershell
$ $Env:PERCY_TOKEN="<your-project-token>"
Step 5: Execute Percy Test
Use the below command to execute your Percy NightwatchJS script.
npx percy exec -- nightwatch <testscript_path>
Example:
npx percy exec -- nightwatch tests/demoVRT.js
Step 6: View The Result
Once the execution is completed, the command line tool provides the output with Percy build URL. You can easily navigate to the build using the build URL to see the visual validation result.
If there are no changes, then Percy shows the message “No Changes”
If there is a difference between the baseline screenshot and the actual screenshot when Percy shows the side-by-side difference
HTML Element Visual Testing using Percy
Percy can also help you to capture and compare individual HTML elements. Instead of capturing the entire page you can capture the snapshot of the required element and perform visual testing.
The scope option in percySnapshot() command can help to capture the HTML element
Example Code:
//demoVRT.js
describe('Nightwatch Visual Test Demo', function () {
it('Nightwatch Visual Test', async function (browser) {
await browser
.navigateTo('https://www.browserstack.com')
.percySnapshot('Single Element Snapshot', { scope: '#product-menu-toggle' })
.end();
});
});
In the above code, we are specifying the selector in the scope option instead of capturing the entire page.
.percySnapshot(‘name’, { scope: ‘#product-menu-toggle’ }) : The HTML element which matches the selector with id product-menu-toggle will be captured and used for Visual Validation.
Percy NightwatchJS Best Practices
Use the Visual Testing command for each web application page
Schedule the build and analyze the result more frequently. This guarantees no accidental changes have been pushed.
Use the diff sensitivity settings, such as strick, sensitive and relaxed mode, which helps to identify the color differences.
Consider testing responsive visual testing, The responsive visual testing ensures there are no visual bugs in different screen resolutions. This can be done in by choosing the available resolution in Percy build dashboard
The Multiple browser testing helps to ensure the page is rendering in all the required browsers without any glitches. Percy allows users to choose multiple browsers in Project settings.
Conclusion
Manual testing is always error-prone whereas the automated way of visual testing saves time and provides the best result. The major problem when you use a different tool is upgrading the skill set, but Percy provides out-of-the-box integration for all major automation frameworks, so the learning curve is very less. Also, using Percy is very easy.
By integrating Percy with NightwatchJS, the single test automation framework can be enhanced to perform both functional and Visual regression testing.