Appium Visual Testing: The Essential Guide

Appium is a free, open-source, cross-platform testing tool for desktop, mobile, and desktop apps. 

• It was first developed to automate iOS and Android mobile apps and has developed into a fully functional platform offering WebDriver-based automation options. 
• It supports app automation across platforms like iOS, Android, and Windows. 
• Since version 2.0, every driver has been isolated from the Appium server apps and can be managed alone using the Appium Driver command line interface.

What is Visual Testing?

Visual testing is a crucial type of testing that may identify both visual defects and functional defects. Although it is generally complementary to functional testing, visual testing is combined.

• Visual testing assesses visual changes to ensure that only deliberate UI changes have been implemented in production. 
• Many businesses mistakenly believe that visual testing is not required while conducting thorough functional tests. 
• Visual testing finds flaws that functional testing misses and verifies that your program looks as it should
• Visual regressions are equally important as functional regressions, especially when they reveal issues that a functional test suite might miss.

Benefits of Visual Testing

No matter how well your application’s functionality works, it will be useless to the consumer unless it can give a decent user interface and user experience. Nowadays, we talk about UI (User Interface) and put a lot of emphasis on UX (User Experience).

• When we give a better user experience, the visual testing suite becomes much more crucial since a more appealing visual design may be recognized as more usable. 
• The benefits of automating visual tests are that they provide long-term cost efficiency, are faster than manual tests, are more precise because they cannot include human issues, and deliver pixel-perfect visual tests.
• They are reusable and clear because they offer automatic reports that are easily and readily accessible by anyone on the team.

Given its importance, businesses should devote substantial time and resources to visual testing. Combined with current developer tests, it will aid in detecting visual problems in the early phases of the development lifecycle.

Why perform Appium Visual Testing?

We know how to take screenshots using Appium. But how do we balance two screenshots to detect visual differences?

A basic approach to the visual distinction would be far too greedy because even condensation algorithms may produce nearly undetectable differences. In the realm of open-source software, OpenCV is an assortment of image manipulation capabilities that can be used without the user needing to comprehend the intricate details of its inner workings. Appium has OpenCV support, but it is not enabled by default since developing OpenCV and its Node.js bindings takes time and must be done on each platform independently. 

For the easiest way to get everything ready for usage with Appium, run npm install -g opencv4nodejs.

Steps for Visual Validation

The primary goal is to snap screenshots of each view we encounter while traveling through our app (maybe during a functional test). We compare these screenshots to a prior screenshot of the identical view using an image analysis application. 

We may have detected a visual regression if there are any significant changes. At that point, we may either reject the test or record the difference in a database for further assessment by the team.

Step 1: In this step, We’ll need to test the application and take screenshots.

Step 2: In this step, These screenshots are compared to the baseline screenshots by the automation tool. Typically, baseline screenshots are photographs a tester has confirmed were taken during previous test sessions.

Step 3: After getting the results of the image comparisons, the application generates a report detailing all the discrepancies observed.

Step 4: In the last step, The tester reviews the report, determining if each difference is a bug or a valid change (false positives). The baseline photographs are updated as a result of this.

You don’t have any baseline photos for the first test run. As a result, photographs from the first run are frequently utilized as baseline images. They are compared to the screenshots starting with the following run.

Visual Comparison Example

We are validating the “$” sign in this image because $ is not having your locator. First, we take a screenshot of the UI and then use OpenCV, to compare the images if the images are not the same then we will raise a bug to the responsible team.

Setting up the Appium Visual Testing Environment

Setup Appium

Download and install Java (JDK) and set a JDK and bin folder path.

• Download the “.exe” file from here (Version: jdk1.8.0_91 or whichever is the latest you find there).
• Install the “.exe” file.
• Setup the JDK bin folder path in your system’s environment variable.

 Download the Android SDK

• Click on the link “android-sdk_r24.4.1-windows.zip” (or whichever is the latest you find there) and click the download button.
•  Once the zip file gets downloaded, unzip the folder.
• Now click on the “SDK Manager.exe” file
• This opens the Android SDK Manager window. Select “Tools” and the Android platform on which you will perform your tests.

Install Appium

• Open the Appium Official link.
• Click on the Download link.
• Select the OS of the machine you are working on and download the appropriate version.
• Unzip the downloaded zip folder.
• Install the .exe file “appium-installer”.

Setting up Appium for visual testing

Appium has support for OpenCV, albeit not enabled by default since developing OpenCV and its Node.js bindings takes time and must be done on specific systems.

• The simplest method to get things ready for usage with Appium is to run npm install -g opencv4nodejs. 
• This will try to install the Node bindings globally and download and build OpenCV on your machine. 
• If that doesn’t work, you may install OpenCV via Homebrew and then install the Node bindings with the OPENCV4NODEJS_DISABLE_AUTOBUILD=1 env flag in front to instruct it to use the system-installed binaries.

Once you’ve installed the opencv4nodejs package, you must ensure it is accessible to Appium when it starts. One method is to execute the npm install command without the -g parameter within the Appium directory. Another option is to include your global node_modules folder in the NODE_PATH environment setting.

Performing Appium Visual Testing

Test Case

Follow the below test steps for visual testing:

1. First we store an image for the app home page as a baseline image.
2. Launch the app using appium cap file. 
3. Wait to upload the homepage
4. Click to Add an item to the cart 
5. Click on the cart button 
6. Go to the cart and verify the cart screen with the baseline image (ScreenShot)

Framework Design and Code

For framework design, follow below structure

Test Script

AppiumVisualTestBrowserStackAPP.java




import io.appium.java_client.MobileBy;

import io.appium.java_client.imagecomparison.SimilarityMatchingOptions;

import io.appium.java_client.imagecomparison.SimilarityMatchingResult;

import java.io.File;

import java.net.URISyntaxException;

import org.apache.commons.io.FileUtils;

import org.junit.Test;

import org.openqa.selenium.By;

import org.openqa.selenium.OutputType;

import org.openqa.selenium.WebElement;

import org.openqa.selenium.remote.DesiredCapabilities;

import org.openqa.selenium.support.ui.ExpectedConditions;

import org.openqa.selenium.support.ui.WebDriverWait;





public class AppiumVisualTestBrowserStackAPP extends BaseTest {




// Give a file path where we can save the matched file

private final static String path_to_validate = "/Users/Download/bs_demo/";




private final static String CHECK_HOME = "home_screen";

private final static String CART_PAGE = "cart_page";

private final static String BASEIMAGE = "BASEIMAGE_";




private final static double Breakpoint_for_Match = 0.99; //Thresold Value




private final static By ADD_TO_CART = MobileBy.AccessibilityId("add-to-cart-12");

private final static By NAV_TO_CART = MobileBy.AccessibilityId("nav-cart");




@Override

protected DesiredCapabilities getCaps() throws URISyntaxException {

DesiredCapabilities capabilities = new DesiredCapabilities();

capabilities.setCapability("platformName", "Android");

capabilities.setCapability("deviceName", "Android Emulator");

capabilities.setCapability("automationName", "UiAutomator2");

capabilities.setCapability("app", getResource("apps/browserstack-demoapp.apk").toString());

//Make sure we uninstall the app before each test regardless of version

capabilities.setCapability("uninstallOtherPackages", "io.cloudgrey.the_app");




return capabilities;

}




private WebElement waitForElement(WebDriverWait wait, By selector) {

WebElement el = wait.until(ExpectedConditions.presenceOfElementLocated(selector));

try { Thread.sleep(750); } catch (InterruptedException ign) {}

return el;

}




@Test

public void testAppDesign() throws Exception {

WebDriverWait wait = new WebDriverWait(driver, 5);




// wait for an element that's on the home screen

WebElement addToCart = waitForElement(wait, ADD_TO_CART);




// now we know the home screen is loaded, so do a visual check

doVisualCheck(CHECK_HOME);




// Click on add to cart btn for adding item in card

addToCart.click();

WebElement navToCart = waitForElement(wait, NAV_TO_CART);




//click to cart btn

navToCart.click();




//Perform our second visual check, this time of the cart page

doVisualCheck(CART_PAGE);

}




private void doVisualCheck(String checkName) throws Exception {

String basematchFilename = path_to_validate + "/" + BASEIMAGE + checkName + ".png";

File basematchImg = new File(basematchFilename);




// If there is no basematch picture for this check, one should be made.

if (!basematchImg.exists()) {

System.out.println(String.format("No basematch found for '%s' check; capturing baseline instead of checking", checkName));

File newBasematch = driver.getScreenshotAs(OutputType.FILE);

FileUtils.copyFile(newBasematch, new File(basematchFilename));

return;

}




// Otherwise, obtain the picture similarity from Appium if we discover a basematch. Obtaining the resemblance

// We also enable visualisation so that, should something go wrong, we can see what went wrong.

SimilarityMatchingOptions opts = new SimilarityMatchingOptions();

opts.withEnabledVisualization();

SimilarityMatchingResult res = driver.getImagesSimilarity(basematchImg, driver.getScreenshotAs(OutputType.FILE), opts);




// If the similarity is not high enough, consider the check to have failed

if (res.getScore() < Breakpoint_for_Match) {

File failViz = new File(path_to_validate + "/FAIL_" + checkName + ".png");

res.storeVisualization(failViz);

throw new Exception(

String.format("Visual check of '%s' failed; similarity match was only %f, and below the breakPoint of %f. Visualization written to %s.",

checkName, res.getScore(), Breakpoint_for_Match , failViz.getAbsolutePath()));

}




// Otherwise, it passed!

System.out.println(String.format("Visual check of '%s' passed; similarity match was %f",

checkName, res.getScore()));

}




}

BaseTest.java

import io.appium.java_client.android.AndroidDriver;

import java.io.IOException;

import java.net.URISyntaxException;

import java.net.URL;

import java.nio.file.Files;

import java.nio.file.Path;

import java.nio.file.Paths;

import java.util.Base64;

import org.junit.After;

import org.junit.Before;

import org.openqa.selenium.remote.DesiredCapabilities;




public class BaseTest {




AndroidDriver driver;




protected DesiredCapabilities getCaps() throws Exception {

throw new Exception("Must use getCaps");

}




@Before

public void setUp() throws Exception {

URL server_url = new URL("http://localhost:4723/wd/hub");

driver = new AndroidDriver(server_url, getCaps());

}




@After

public void tearDown() {

if (driver != null) {

driver.quit();

}

}




Path getResource(String file_name) throws URISyntaxException {

URL ref_img_url = getClass().getClassLoader().getResource(file_name);

return Paths.get(ref_img_url .toURI()).toFile().toPath();

}




private String getResourceB64(String file_name) throws URISyntaxException, IOException {

Path ref_img_path = getResource(file_name);

return Base64.getEncoder().encodeToString(Files.readAllBytes(ref_img_path ));

}




String getReferenceImageB64(String file_name) throws URISyntaxException, IOException {

return getResourceB64("images/" + file_name);

}




}

You can run this script or test directly to the main class.

Result Analysis 

After the first execution, We save the images in the baseline for the app’s homepage and after adding items to the cart. 

• We put a threshold value in the code for the comparison of images.
• If the actual getScore value exceeds the threshold, the test case fails. 
• If the getScroe value is less than the threshold value, then the test case is getting passed.

private final static double Breakpoint_for_Match= 0.99;

if (res.getScore() < Breakpoint_for_Match) {

File failViz = new File(path_to_validate + "/FAIL_" + checkName + ".png");

res.storeVisualization(failViz);

throw new Exception(

String.format("Visual check of '%s' failed; similarity match was only %f, and below the Breakpoint_for_Match of %f. Visualization written to %s.",

checkName, res.getScore(), Breakpoint_for_Match, failViz.getAbsolutePath()));}

// Otherwise, it passed!

System.out.println(String.format("Visual check of '%s' passed; similarity match was %f",

checkName, res.getScore()));

Image Comparison Commands

When you run this command with the appropriate image byte arrays (in this example, img1 is our baseline and img2 is the snapshot we want to investigate), and the right options object (it must have a version of SimilarityMatchingOptions), a SimilarityMatchingResult object is produced.

• The most important function of the result object is getScore, which will tell us how comparable the two photographs we submitted were in terms of a score between 0 and 1. 
• We just assess whether the score is greater than a predefined threshold to translate this into genuine validation (which should be selected experimentally based on your app). 
• If so, we don’t believe there are any discernible changes. If not, we may issue an exception, causing the test to fail, or find another way to indicate the difference.

SimilarityMatchingResult res = driver.getImagesSimilarity(baselineImg, driver.getScreenshotAs(OutputType.FILE), opts);

Best Practices for Appium Visual Testing

• For the tool to pass your testing, it must be able to handle anti-aliasing, pixel offsets, etc.
• Release faster with DOM snapshotting and advanced parallelization capabilities designed for executing complex test suites at scale
• Review, collaborate, and approve snapshots, keeping the rest of the team updated throughout the process
• Choose a tool like Percy that can ignore false positives in visual testing.
• Test automation should be capable of dealing with dynamic and shifting information.
• Don’t rely on threshold settings or error ratios. The only thing that should be important is if a person can tell the difference and whether it will affect how the user interacts with the product.
• The automation program must be able to evaluate the page’s structure and perform layout comparisons.
• Validate the full UI page rather than individual parts. More comprehensive coverage will result from this. You risk missing unexpected issues if you validate only particular components.
• Put the correct image data in the framework to ignore more thresholds.

Get Started with Automated Visual Testing

We’ve seen how to use Appium’s built-in image processing technology (powered by OpenCV) to build out some basic visual testing capabilities. In this article, we have complete function testing using visual testing. We perform small operations, but you can create a framework using this article.