How to release on time without the risk of sending costly bugs to production
Compliance laws have impacted the AdTech and MarTech industries and business models, influencing how publishers, advertisers and consumers interact.
As the world’s largest media investment company, GroupM powers 1 in every 3 agency network ads in the world, making $45 billion in annual billings. Staying on top of changing regulations and adjusting to meet compliance requirements is critical to thrive in the advertising industry, and noncompliance with local regulations can result in significant fines.
GroupM needed to ensure [m]Platform was compliant with local regulations. [m]Platform is GroupM’s proprietary advertising technology which harnesses consumer data from hundreds of sources to provide insights to serve targeted programmatic ads based on data science and machine learning.
“This compliance deadline was critical for us. We had to upgrade our systems, software and product, and do a production rollout for [m]Platform in a very short timeframe,” said GroupM’s Global Head of Quality Assurance, Moshiur Khondokar.
The initiative included releasing a customer-facing web application to [m]Platform’s large user base. Khondokar and his QA team had to ensure the application worked for every customer, no matter their mode or location of access. The challenge was ensuring everything worked glitch-free for a broad audience in precisely 4 weeks.
“The website was simple to build, but given the high impact on the bottom line, there was no room for error. A minor bug could flag noncompliance, and potentially expose the company to millions if not billions in fines.”
It was only recently that the QA team at GroupM had kicked off test automation, under the vision of GroupM CTO Bob Hammond. “With our existing internal setup, it was difficult to deliver on time. We would’ve had to manually configure every browser, platform and geolocation setting, which is a big hassle. Since we have only a few QA automation engineers and a rapid release cycle from ideation to go-live, this was a pretty hectic task.”
The stakes of the launch were high. The senior leadership at GroupM recognized the need to scale their cross-browser testing to mitigate any risk of a serious bug getting to production. In 4 weeks, GroupM had to do everything from picking the right solution to testing and debugging the website before releasing it to production.
High performance infrastructure offering multiple browsers, platforms and geolocation
With this critical deadline approaching in 4 weeks, Khondokar had to figure out how to test the [m]Platform registry for every user scenario before launching it.
“I wanted an out-of-box solution where GroupM could run automation code concurrently in multiple browsers, multiple platforms and multiple locations. We had been using Selenium WebDriver and Docker to test a single scenario at a time, and were handicapped by the limited number of configurations we could initiate internally. With BrowserStack, I could choose from thousands of configurations, run test scenarios in parallel, and be sure that my applications worked on all of them.”
Pre-evaluation, GroupM had been leaning toward a competing product. However, once Khondokar evaluated potential solutions, BrowserStack turned out to be the only provider meeting the business requirements, outperforming the competitor on three key criteria: speed, reliability and cost.
“The reason we chose BrowserStack is its geo-proximity of data centers to major business hubs. The competing product did not have distributed data centers, so our tests were running very slowly. We wanted to run URLs from IP addresses in different locations and fire ad tags to see how they behaved. We weren’t able to do this with the competing product.” In addition to the speed of testing, BrowserStack got a stamp of approval for its SOC 2 security compliance and accuracy of test results. Considering GroupM works with leading enterprise advertisers, the security of their customer data was of utmost importance to the organization.
Compared to the competitor, “BrowserStack gives real devices at a price the competitor offers virtual devices (emulators). The performance and accuracy of results are completely different when testing on a real device versus a virtual device.”
Fast, accurate testing leads to quicker decisions, lower risks and on-time delivery
Using BrowserStack, Khondokar and his team were able to deliver the [m]Platform Registry on time without releasing any serious bugs that could have risked their regulatory compliance. Instead of waiting until production, they were able to find and fix issues pre-release.
“Previously, debugging was a nightmare. The QA team had difficulties finding the root cause when a test failed. Since we have multiple web services/APIs, we would have to search in multiple places, like the production box, application logs, server logs and Splunk logs. With BrowserStack, we now have video recorded sessions to quickly trace back the exact step where a test fails. Finding the root cause is much more convenient for QA engineers. It makes the difference of debugging in minutes versus multiple days.”
Khondokar and his team are already seeing positive results across quality assurance workflows, from overnight hotfixes to regression testing. “Before BrowserStack, functional regression tests used to take at least a week’s effort for 4 manual QA engineers. Now with BrowserStack Automate, our builds run in just an hour. If we find issues, we talk about the risks and make a decision”
“BrowserStack has changed how we execute our tests, how we publish our test results, and how we make decisions.” Khondokar looks to the future of scaling with BrowserStack, “We want to get to a point where complete test automation takes a few minutes, where we can make instant decisions and implement CI/CD, and thereby deliver quality software products.”