GoodRx Cuts Testing Time by 90% to Release 15 Times a Day

GoodRx automates web and mobile app testing with BrowserStack, scaling from 0 to 2.7 million tests in 1 year.

GoodRx's usage of BrowserStack
GoodRx’s usage of BrowserStack

THE CHALLENGE

Going from no QA to fully operational QA in 8 months

From an office nestled in Santa Monica, California, a unicorn startup leads the fight against high prescription drug costs in America, saving the everyday consumer up to 80% on medications. GoodRx serves more than 10 million customers a month, providing a tool to compare prescription drug prices between pharmacies, find discount coupons and rely on for affordable healthcare.

Operating in a heavily regulated healthcare industry, GoodRx must meet compliance requirements, including clearly displaying drug information to consumers. Bugs like missing fields and incorrect information could flag noncompliance and lead to severe consequences. “Even the smallest bugs in production can cost hundreds of thousands of dollars,” says Priyanka Halder, Senior Manager of Quality Engineering.

Both consumers and pharmacies rely on GoodRx’s web and mobile applications as sources of healthcare and revenue, respectively. The apps must work seamlessly every time, which their Quality Engineering team ensures. In fact, GoodRx is the #1 medical app for iOS and Android.

While the team’s process works like clockwork today, this wasn’t always the case. 

Before the shift to quality engineering, “it was pretty challenging to do QA when there was one tester among 40 developers,” says Priyanka. And without a defined process in place, it was an ad hoc process where tickets were raised as issues arose, leading to bottlenecks and in critical cases, missed bugs and diminishing release quality. Furthermore, QA environments and production environments weren’t identical, so the QA team couldn’t replicate realistic scenarios to test on. Not to mention, manual quality assurance created redundancy and inefficiency.

To tackle GoodRx’s problems, Priyanka was tasked to create an automated QA process, build the team, adopt the right tools—and eventually scale the operation.

THE SOLUTION

Implementing an effective test automation setup that scales easily

First, Priyanka built a distributed Quality Engineering team with 24/7 support. The 8-person team is divided into 3 functions: QA Engineers take care of day-to-day releases, SDETs work on tooling and automation, and QA testers run tests to expedite the release of a new feature.

Next, Priyanka and her team set up hourly automation tests on BrowserStack to sanity check the site, starting with the P0/P1 pipeline. “We segregate our priorities,” she explains. “The P0/P1 is a continuous pipeline that always needs to be working and cannot be broken. [Any problems] need to get sorted instantly. When a P0/P1 feature makes it to production, our pipeline already has those tests ready to be run.”

GoodRx adopts ‘Shift Right’ to mitigate risk
GoodRx adopts ‘Shift Right’ to mitigate risk

With a solid pipeline in place, the QA team of GoodRx is able to ‘Shift Right’, and move testing right into production. ‘Shift Right’ testing is best suited for the high-velocity, high frequency releases the company sees. Some new releases that are directly pushed to production are ‘feature flagged’. Dogfooding routes internal users to the new features before releasing to the public. With this approach, the team is able to mitigate risk by finding new and unexpected usage scenarios—and ship fast.

“BrowserStack has been one of our biggest partners for [test automation] success,” says Priyanka. GoodRx is now able to test deterministically and increase test coverage to prevent P0/P1 mishaps. It’s all about “choosing the right tool,” she continues, “It takes time and a good tool to build the right solution.”

THE IMPACT

Using BrowserStack, GoodRx has reduced regression testing time by 90% and cut test execution time to 5 minutes.

GoodRx went from no test automation to running 100 concurrent tests within months. “We rely heavily on parallelization since we release 10-15 times a day. Without BrowserStack, it would take each person 6-7 hours to manually test our pages on multiple browsers. Right now, it takes less than five minutes.”

“Today, every feature we release goes through test automation first. We’ve run 2.4 million tests on BrowserStack so far. Even the smallest change is run through different browsers to ensure everything is stable.” GoodRx maintains 99.9% uptime. “And if anything fails, we use the BrowserStack API to check what exactly failed and why. Developers are happy they don’t have to dig through code to see what failed. They can just reproduce the bug on BrowserStack Live and find failures instantly.”

One fine day, Priyanka got a notification on her phone at 4 AM. “I got pinged about a test fail on Slack. I was able to instantly access the BrowserStack dashboard to check on the test. It was nothing serious thankfully, but it’s reassuring to have my BrowserStack test results integrated with Slack.”

“We use BrowserStack heavily for cross browser and mobile web testing automation. Everybody here uses it. BrowserStack has changed the way developers think: they no longer feel QA is a blocker. Our tests are now super solid—they catch anything and everything. We always have the assurance that quality is taken care of, and see a lot of value in this product. I can’t be happier to vouch for it.”

Watch the webinar to hear the story directly from Priyanka Halder, Sr. Manager, Quality Engineering.

NUMBER OF EMPLOYEES

200+

FOUNDED

2011

INDUSTRY

Healthcare Technology

BROWSERSTACK PRODUCTS USED

Automate
Live
App Automate
App Live

ROI

Regression testing time reduced by 90%

"We rely heavily on parallelization since we release 10-15 times a day. Without BrowserStack it would take each person 6-7 hours to manually test our pages on multiple browsers. Right now, it takes less than five minutes."
Priyanka Halder
Senior Manager of Quality Engineering, GoodRx

GoodRx Cuts Testing Time by 90% to Release 15 Times a Day

GoodRx office
“We use BrowserStack heavily for cross-browser and mobile test automation. Our tests are now super solid—they catch anything and everything. We always have the assurance that quality is taken care of, and see a lot of value in this product.”
Priyanka Halder
Priyanka Halder Delivery Lead, GoodRx
Industry
Healthcare Technology
Location
California, United States
Ready to try BrowserStack?
Join over 2M developers & 50K teams across 135 countries

Introduction

GoodRx, a tech-driven digital healthcare marketplace, is on a mission to help millions of Americans find affordable and convenient healthcare. The everyday consumer can save up to 80% on medication using GoodRx’s free-to-use website and mobile app, which incidentally is the most downloaded medical app on the iOS and Android app stores.

While the team’s QA process works like clockwork today, this wasn’t always the case. BrowserStack enabled the newly built team to transition from manual to automated web and mobile app testing in under a year. As a result, they scaled from 0 to 2.7 million automated tests in 1 year, reduced testing time by 90%, and improved release quality significantly.

GoodRx team
The challenge

Setting up a QA process for test automation success

Operating in a heavily regulated healthcare industry, GoodRx must meet compliance requirements, including clearly displaying drug information to consumers. Bugs like missing fields and incorrect information could flag non-compliance and lead to severe consequences. In addition, consumers and pharmacies rely on GoodRx’s web and mobile applications as sources of healthcare and revenue, respectively.

“Even the smallest bugs in production can cost GoodRx hundreds of thousands of dollars,” says Priyanka Halder. The GoodRx Quality Engineering team must ensure that the website and mobile app work flawlessly every single time.

Before the shift to quality engineering, “it was pretty challenging to do QA when there was one tester among 40 developers,” says Priyanka. And without a defined process in place, it was an ad hoc process where tickets were raised as issues arose, leading to bottlenecks and in critical cases, missed bugs and diminishing release quality.

In addition, QA environments and production environments weren’t identical, so the team was unable to replicate realistic scenarios to test on. Heavy dependency on manual testing made the QA process time-consuming, inefficient, and stressful.

To tackle GoodRx’s problems, Priyanka was tasked to create an automated QA process, build the team, adopt the right tools—and eventually scale the operation.

“We rely heavily on parallelization since we release 10-15 times a day. Without BrowserStack, it would take each person 6-7 hours to manually test our pages on multiple browsers. Right now, it takes less than five minutes.”
Priyanka Halder
Priyanka Halder, Delivery Lead, GoodRx
The solution

Empowering the team with the right tools

Priyanka started off by building a distributed Quality Engineering team with 24/7 support. The 8-person team is divided into 3 functions: QA Engineers take care of day-to-day releases, SDETs work on tooling and automation, and QA testers run tests to expedite the release of a new feature.

She and her team also set up hourly automation tests on BrowserStack to sanity check the site, starting with the P0/P1 pipeline. “We segregate our priorities,” she explains. “The P0/P1 is a continuous pipeline that always needs to be working and cannot be broken. [Any problems] need to get sorted instantly. When a P0/P1 feature makes it to production, our pipeline already has those tests ready to be run.”

Today, every feature that GoodRx releases goes through test automation first. “We’ve run 2.4 million tests on BrowserStack [in 8 months]. Even the smallest change is run through different browsers to ensure everything is stable. And if anything fails, we use the BrowserStack API to check what exactly failed and why. Developers are happy they don’t have to dig through code to see what failed. They can just reproduce the bug on BrowserStack Live and find failures instantly,” she explains. GoodRx is now able to test deterministically and increase test coverage to prevent P0/P1 mishaps.

It’s all about “choosing the right tool,” she continues. “It takes time and a good tool to build the right solution.”

The impact

A culture of quality with 90% reduction in testing time

“BrowserStack has been one of our biggest partners for [test automation] success,” says Priyanka. “It has changed the way developers think: they no longer feel QA is a blocker.”

Within months of using BrowserStack, GoodRx went from no test automation to running 100 concurrent tests. “We rely heavily on parallelization since we release 10-15 times a day. Without BrowserStack, it would take each person 6-7 hours to manually test our pages on multiple browsers. Right now, it takes less than five minutes.”

“We use BrowserStack heavily for cross-browser and mobile web testing automation. Our tests are now super solid—they catch anything and everything. We always have the assurance that quality is taken care of, and see a lot of value in this product,” she explains.

Priyanka also appreciates how BrowserStack easily integrates with the existing workflow at GoodRx – the Slack integration, for example. One fine day, she got a Slack notification on her phone at 4 AM about a failed test. “I was able to instantly access the BrowserStack dashboard to check on the test. It was nothing serious thankfully, but it’s reassuring to have my BrowserStack test results integrated with Slack.”

Watch the webinar to hear the story directly from Priyanka Halder, Delivery Lead at GoodRx.

“BrowserStack has been one of our biggest partners for [test automation] success.”
Priyanka Halder
Priyanka Halder, Delivery Lead, GoodRx

What will your team do with BrowserStack?

Over 2M developers & 50K teams already test on BrowserStack. Join them