Going from no QA to fully operational QA in 8 months
From an office nestled in Santa Monica, California, a unicorn startup leads the fight against high prescription drug costs in America, saving the everyday consumer up to 80% on medications. GoodRx serves more than 10 million customers a month, providing a tool to compare prescription drug prices between pharmacies, find discount coupons and rely on for affordable healthcare.
Operating in a heavily regulated healthcare industry, GoodRx must meet compliance requirements, including clearly displaying drug information to consumers. Bugs like missing fields and incorrect information could flag noncompliance and lead to severe consequences. “Even the smallest bugs in production can cost hundreds of thousands of dollars,” says Priyanka Halder, Senior Manager of Quality Engineering.
Both consumers and pharmacies rely on GoodRx’s web and mobile applications as sources of healthcare and revenue, respectively. The apps must work seamlessly every time, which their Quality Engineering team ensures. In fact, GoodRx is the #1 medical app for iOS and Android.
While the team’s process works like clockwork today, this wasn’t always the case.
Before the shift to quality engineering, “it was pretty challenging to do QA when there was one tester among 40 developers,” says Priyanka. And without a defined process in place, it was an ad hoc process where tickets were raised as issues arose, leading to bottlenecks and in critical cases, missed bugs and diminishing release quality. Furthermore, QA environments and production environments weren’t identical, so the QA team couldn’t replicate realistic scenarios to test on. Not to mention, manual quality assurance created redundancy and inefficiency.
To tackle GoodRx’s problems, Priyanka was tasked to create an automated QA process, build the team, adopt the right tools—and eventually scale the operation.
Implementing an effective test automation setup that scales easily
First, Priyanka built a distributed Quality Engineering team with 24/7 support. The 8-person team is divided into 3 functions: QA Engineers take care of day-to-day releases, SDETs work on tooling and automation, and QA testers run tests to expedite the release of a new feature.
Next, Priyanka and her team set up hourly automation tests on BrowserStack to sanity check the site, starting with the P0/P1 pipeline. “We segregate our priorities,” she explains. “The P0/P1 is a continuous pipeline that always needs to be working and cannot be broken. [Any problems] need to get sorted instantly. When a P0/P1 feature makes it to production, our pipeline already has those tests ready to be run.”
With a solid pipeline in place, the QA team of GoodRx is able to ‘Shift Right’, and move testing right into production. ‘Shift Right’ testing is best suited for the high-velocity, high frequency releases the company sees. Some new releases that are directly pushed to production are ‘feature flagged’. Dogfooding routes internal users to the new features before releasing to the public. With this approach, the team is able to mitigate risk by finding new and unexpected usage scenarios—and ship fast.
“BrowserStack has been one of our biggest partners for [test automation] success,” says Priyanka. GoodRx is now able to test deterministically and increase test coverage to prevent P0/P1 mishaps. It’s all about “choosing the right tool,” she continues, “It takes time and a good tool to build the right solution.”
Using BrowserStack, GoodRx has reduced regression testing time by 90% and cut test execution time to 5 minutes.
GoodRx went from no test automation to running 100 concurrent tests within months. “We rely heavily on parallelization since we release 10-15 times a day. Without BrowserStack, it would take each person 6-7 hours to manually test our pages on multiple browsers. Right now, it takes less than five minutes.”
“Today, every feature we release goes through test automation first. We’ve run 2.4 million tests on BrowserStack so far. Even the smallest change is run through different browsers to ensure everything is stable.” GoodRx maintains 99.9% uptime. “And if anything fails, we use the BrowserStack API to check what exactly failed and why. Developers are happy they don’t have to dig through code to see what failed. They can just reproduce the bug on BrowserStack Live and find failures instantly.”
One fine day, Priyanka got a notification on her phone at 4 AM. “I got pinged about a test fail on Slack. I was able to instantly access the BrowserStack dashboard to check on the test. It was nothing serious thankfully, but it’s reassuring to have my BrowserStack test results integrated with Slack.”
“We use BrowserStack heavily for cross browser and mobile web testing automation. Everybody here uses it. BrowserStack has changed the way developers think: they no longer feel QA is a blocker. Our tests are now super solid—they catch anything and everything. We always have the assurance that quality is taken care of, and see a lot of value in this product. I can’t be happier to vouch for it.”
Watch the webinar to hear the story directly from Priyanka Halder, Sr. Manager, Quality Engineering.