Cross Browser Compatibility Testing beyond Chrome & Firefox
By Akshay Pai, Community Contributor and Praveen Umanath, Director of Product - May 17, 2019
Cross browser compatibility testing has been gaining a lot of traction in recent years and there is a reason for it. While technology is evolving rapidly, people aren’t. A significant amount of people are resistant to changes, or more specifically, “have an aversion to upgrading their tech”. In this scenario, it’s browser compatibility testing that enables companies to ensure that no customer is left behind or has an experience that is not desired. So even though browsers like Google Chrome and Firefox dominate the market, there are people using their older versions, or other browsers. And their numbers are too high to be ignored.
What is cross browser compatibility testing?
Cross browser compatibility testing is a non-functional form of testing, which emphasizes on availing your website’s basic features and functionality to users on different browser-OS combinations, devices, and assistive tools.
How does it impact your application?
Not all browsers and devices work on the same configuration; they face browser compatibility issues on different levels. This inconsistency is the reason why you might observe the lack of application uniformity across browsers and devices. You would not want a section of your prospective users to not be able to access the application features.
That is what makes cross browser testing important. If your website is not tested and debugged on different platforms and browsers, it won’t work the same on all of them, causing inconvenience to the users, subsequently impacting your business.
Which browsers to choose for cross browser testing?
Since it’s impossible to test on every possible browser-device combination, you need to shortlist the most important ones to test your web application on. As of December 2018, Google Chrome has the largest number of users. It accounts for about 70.95% of the market. Firefox comes second with a market share of 10.05%, while others such as IE, Safari, and Edge has a market share of approx 4-5% each.
Given the volume of users, Google Chrome and Firefox are critical for every business. However, browsers such as Safari, Edge, and Internet Explorer should also be a part of the testing and a subset of them can be chosen based on your traffic.
Is volume and market share sufficient enough to decide the browsers to test on?
The answer is No. Firstly, basing the decision solely on volume would leave out a number of browsers in the market. Your target market should be considered as well. For example, Windows XP is still being used by millions of people, and those users are likely to use Internet Explorer. If your target market is using IE, then your website should be compatible with IE.
Other crucial factors to shape this decision could be business indicators and value addition. Most Safari users run it on Apple devices, which is a reasonable indicator of high spending power. This makes Safari a significant browser for testing, despite its relatively low user volume (compared to Chrome and Firefox).
Eventually, a combination of market share, value, and the nature of your web app will help you shortlist the browsers your website needs to be compatible with.
Are browsers like Edge, Safari, and IE still relevant for cross browser compatibility?
Yes, surprisingly they are! Let’s analyze the market share of these browsers in different countries to understand how each of them is relevant, despite low global usage statistics.
Safari is mostly used among the Apple iPhones, Mac and iPad users, who can spend considerably higher than users of other devices. This could be relevant to luxury brands, for instance.
IE and Edge are developed by Microsoft, which is still the preferred platform among users who prioritize its familiarity, the corporates who rely on Microsoft Office tools, and more.
Given the fact that Edge was launched with Windows 10, plenty of people using previous versions of Windows still use IE, despite the lack of performance and features. Edge is yet to emerge in terms of market share with newer versions of Windows OS.
Let’s look for specific markets such as Germany, UK, and the US to determine if browsers like Safari, IE, and Edge are significant in more ways than we think.
Browser usage pattern in Germany
Germany’s market share of web browsers in 2018 shows Google Chrome to be the most used web browser followed by Safari and Firefox. IE also forms a considerable market share of 5.7%. Edge however still lags behind with a 4.04% and is expected to increase with time as IE turns obsolete with newer versions of Windows.
Interestingly, Samsung Internet browser holds 5th place in Germany, with a slightly higher market share than Edge. For businesses who have a market in Germany, compatibility with this browser would be important.
Browser usage pattern in the UK
In UK, Chrome remains at top. But Safari and IE both surpass Firefox, with Edge coming at 5th in terms of market share. It goes on to show how gnoring certain browsers based on global market share could affect the bottom line for businesses operating in different regions.
Browser usage pattern in the US
Coming to the US, Chrome, Safari, and Firefox have a noteworthy presence. However, UC Browser, a popular browser in Asia-Pacific, is gaining turf as people gain awareness of digital advertising and the threats it could pose to data privacy.
After analyzing region-specific browser market shares, it’s clear that IE, Edge, and Safari are still relevant for cross browser compatibility. It boils down to your business’ target audience. Company policies can also influence the selection of browsers. While private organizations are at liberty to cherrypick which browsers they support, govt. portals don’t. They need to be compatible with every browser so everyone in the public domain has access to the information and services.
Finding the right balance
It is important to pick the right combination of web browsers and operating systems to test on. There is usually a trade-off between cost (of testing) and reach. Thus, this entire exercise is to find the right balance—testing on minimum browser/devices that’d help ensure that maximum number of people can use the application.
How to perform cross browser testing keeping feasibility in mind?
Once you’ve made the right choice of browsers based on popularity and audience, you can start cross browser testing manually or with automation tools like Selenium, which is powerful, robust, and trusted by most organizations in the automation testing space.
Considering feasibility and cost, you could also use cloud-based testing infrastructure such as BrowserStack instead of buying and maintaining a lab of real devices to perform cross browser testing.
In a nutshell
Given the disparity in the implementation of open web standards by different web browsers, web developers end up with the task of finding the mid-way of compatibility through cross browser testing. It’s the only way you can deliver a consistent end-user experience across different browsers and devices.