Autodesk Accelerates Release Cycles and Ensures Visual Perfection with BrowserStack AI- Powered Percy

Industry

Software Development

Location

San Francisco, CA

Products

Introduction

In the fast-paced world of software development, where a single bug can lead to significant project delays and financial losses, Autodesk faced the critical challenge of ensuring flawless software releases. Zohar Liran, Director, Head of Engineering, Construction Platform at Autodesk, recognized the imperative of robust end-to-end testing to maintain user trust and operational efficiency. This case study explores how Autodesk, a global leader in 3D design, engineering, and entertainment software, in partnership with BrowserStack, transitioned from fragmented testing practices to a highly automated, visually precise, and confidence-inspiring testing strategy.

The Challenge

Ensuring Quality and Speed in a Complex Environment

Autodesk develops software used by millions worldwide to design, engineer, and create everything from buildings and products to visual effects for films. The complexity and widespread use of their software necessitate an equally robust and reliable platform. Before their partnership with BrowserStack, Autodesk grappled with several testing hurdles:

Reliance on Manual QA: Many teams were still heavily reliant on manual functional and visual testing, leading to slower release cycles and a higher risk of defects reaching production.
Browser and Platform Fragmentation: Supporting multiple browsers, operating systems (macOS, Windows), and mobile platforms introduced significant testing overhead and inconsistencies.
Inefficient End-to-End Testing: Running comprehensive end-to-end tests across various user permissions and project configurations was time-consuming and prone to errors.
Component Library Validation: Ensuring visual consistency and integrity across a vast library of reusable UI components was a significant challenge, especially when changes in one component could inadvertently affect others.
Bridging the Gap Between Engineering and QA: A philosophical divide sometimes existed between development and QA teams, hindering a unified approach to quality assurance.
Test Flakiness and Brittleness: Existing automated tests occasionally suffered from flakiness, undermining confidence in the test suite’s reliability.

With Percy’s AI, you could just click it… there will be an AI prompt to tell you the differences in words. So that again reduces the time that developers have to spend deciphering test results.

Zohar Liran Director, Head of Engineering, Construction Platform at Autodesk

The solution

A Unified Automated Testing Strategy with BrowserStack AI-Powered Percy

Autodesk embarked on a journey to modernize its testing strategy, driven by a commitment to automation and a “fail-fast, fix-fast” culture. The key to their success was a strategic partnership with BrowserStack, leveraging its comprehensive testing platform and Percy for visual regression testing.

Full Automation for End-to-End Testing: Autodesk prioritized automating all possible tests, recognizing that as new features were released, a continuously running, comprehensive test suite would prevent regressions.
BrowserStack for Parallel Testing: By utilizing BrowserStack’s infrastructure, Autodesk could run thousands of tests in parallel across various browsers (including different versions) and operating systems. This drastically reduced test execution times from several minutes to under a minute for many suites.
Percy for Visual Regression Testing: Integrated seamlessly into their CI/CD pipeline, Percy provided pixel-to-pixel visual comparisons. This ensured that beyond functional correctness, the application’s visual elements remained consistent across different resolutions and browsers, catching subtle UI discrepancies.
Cypress Integration: The adoption of Cypress, an open-source testing framework, alongside BrowserStack’s support for it, provided a powerful, fast, and flexible tool for writing and executing tests.
Automated PR Checks: A strict policy was implemented where Pull Requests (PRs) could only be merged if all end-to-end and visual tests passed across all officially supported browsers and environments.
Component Library Visual Testing with Percy: For their Storybook-based component library, Percy was used to automatically generate and validate snapshots of all components. This ensured that any change, even in a nested component, didn’t introduce unintended visual regressions across the entire library.
Data-Driven Coverage Matrix: Autodesk developed a comprehensive spreadsheet-based matrix to map out product flows, user permissions, and different account types. This ensured systematic test coverage across all critical user journeys and configurations.
Treating Tests as First-Class Citizens: A philosophical shift encouraged engineers to consider writing tests as an integral part of feature development, similar to ensuring a car starts after an engine repair. This improved code reviews and overall quality.
Dedicated Foundation Building: For teams transitioning from manual testing, a phased approach was adopted. Small teams were allocated to build a solid foundational test suite, starting with core components and flows, before scaling automation across the entire team.
Emphasis on Flakiness Resolution: Any flaky or brittle tests were treated with high priority. Autodesk collaborated with BrowserStack’s technical team to diagnose and resolve flakiness, ensuring the reliability and trustworthiness of their automated test suite.

A test could come out true, but the screen does not look anything like you want it to look. That’s where visual testing with Percy comes in.

Zohar Liran Director, Head of Engineering, Construction Platform at Autodesk

The impact

Enhanced Quality, Speed, and Confidence

The partnership with BrowserStack and the adoption of a fully automated testing strategy yielded significant benefits for Autodesk:

Accelerated Release Cycles: Test execution times were dramatically reduced, allowing for more frequent and confident releases—potentially three times a week. This significantly improved the speed of new feature delivery and hotfixes.
Reduced Incidents and Bugs: By catching functional and visual bugs earlier in the development cycle, the number of defects reaching production was substantially minimized. This directly translated to a better user experience and reduced operational costs associated with bug resolution.
Increased Developer Confidence: Engineers gained immense confidence in their releases, knowing that comprehensive test suites had validated every change. This “assurance level” liberated them from manual checks and allowed them to focus on innovation.
Improved Product Stability: The rigorous testing, especially with visual validation, ensured that the application remained stable and visually consistent across various platforms and configurations, even with continuous development.
Streamlined Collaboration: The integration of tests into the PR review process provided reviewing engineers with a clearer understanding of changes, fostering better collaboration and faster reviews.
Enhanced ROI on Testing Efforts: The relatively low overhead of writing automated tests compared to the significant benefits in terms of reduced incidents, improved user satisfaction, and faster releases provided a compelling return on investment.
Strong Partnership with BrowserStack: The direct and supportive technical engagement with BrowserStack proved invaluable, providing expert guidance and a willingness to adapt to Autodesk’s specific needs.

With Percy, Autodesk has transformed its visual testing from a time-consuming, error-prone manual effort into an efficient, automated, and AI-powered process that directly contributes to faster, more confident, and higher-quality software releases.

The overall compounded impact of AI in Percy is transformation and drastic. You’re getting your money’s worth and at the end of the day, you’re net positive on the investment.