Regression Testing: A Detailed Guide
By Shaumik Daityari, Community Contributor and Shreya Bose, Technical Content Writer - September 9, 2019
Introducing change in a large, coherent code base is often tricky. A new functionality, bug fix or enhancement may disrupt the functionality of the current version of an app, web application or website. While automated tests like unit tests help developers you solve this problem, this detailed guide on regression testing aims to describe an alternate, more comprehensive route.
What is Regression Testing?
Regression testing is a type of testing in the software development cycle that runs after every change to ensure that the change introduces no unintended breaks.
Regression testing addresses a common issue that developers face — the emergence of old bugs with the introduction of new changes. If a project does not implement strict version control systems, it will be difficult to trace which change introduced a bug.
Therefore, it is a good practice to incorporate robust regression testing in any project.
Why is Regression Testing important?
Typically, it involves writing a test for a known bug and re-running this test after every change to the code base. This aims to immediately identify any change that reintroduces a bug.
With a push for agility in software development, there is an emphasis on adopting an iterative process – push new code often and break things if necessary. Regression testing ensures that with frequent pushes, developers do not break things that already work. The illustration below emphasizes the importance of regression testing.
Regression Testing Techniques
If one is considering the introduction of regression testing into a mature project, one does not need to implement it for every element in the project from the beginning. Various regression testing techniques may be employed.
- A Unit Regression Testing approach uses a bird’s-eye view philosophy to test code. This is a simple method in which the tester has a list of items to test every time a change occurs. This is the best way to start regression testing in an existing project
- A Partial Regression Testing approach divides the project into logical, coherent units that work together to form the whole application. Select the units that are most critical to the application and define specific test cases for them, while performing unit regression testing for the rest of the modules
- The final technique is a Complete Regression Testing, which is the most detailed form of regression testing. In this case, one takes a comprehensive view of the codebase to identify all functionalities that would affect usability on breaking and write detailed tests for each of them. This technique is time-consuming but highly beneficial if applied from the early stages of project development
How to Define A Regression Test Case?
Let us take an example of how to define a regression test.
Imagine a scenario in which the tester would like to test how the shopping cart information appears on the header. The QA needs to answer the following questions:
- Is the number of items in the cart calculated correctly based on the current status? To explore this, one should explore if the back end fetches the correct number of items from the database or local storage.
- Is the variable that stores the number of cart items transferred to the DOM correctly? Make sure that the variable from the backend is correctly sent to the front end.
- Does this number appear on your page when the web page loads?
- Is the element still visible when scrolling down? Does it stay at the top if the header is sticky?
- Is the number updated on adding more items? This involves correctly mapping the action of adding an item to the cart to the DOM element that displays this number.
- Is the color of the element the same? Has it been overwritten by a different element?
- When defining tests for each of these scenarios, it is essential to take a holistic view of all environmental aspects of the variables and elements associated with the shopping cart information on the header.
It is important to remember that regression tests are only as reliable as the details in the test cases. Therefore, it is imperative to devote sufficient time to think about what could go wrong with any element when designing test cases.
Challenges in Regression Testing
While regression testing may be integral to the quality assurance process, it comes with its fair share of challenges. Developers and decision-makers must consider the following before adopting it in their software development cycle.
- Time and Cost Requirements
We have seen that regression testing is incremental — the number of tests changes with each new feature that is added. It involves running older tests with every change. Thus, these tests can take significant time to complete.
One way around this is to adopt parallel tests with newer changes. This lowers the time required to run the tests but may result in higher costs, thanks to added server resources or more people involved.
When a project grows, not only does the number of tests increase but the complexity too. In the shopping cart example above, the testing of the cart is logically done only after creating a new user and browsing to a search page. Thus, the sequence of the tests into play too. With an increasing number of tests, the sequential aspect may lead to a branching out of tests that might become difficult to handle without additional testers.
- Regular Maintenance
To manage the time and complexity of regression testing, one needs to make sure that proper maintenance is done. For example, if a design change mandates the movement of the shopping cart to the right menu — how does that affect the existing regression tests? Though the change may only involve minor changes to the front end, the tests would need to be redesigned completely.
Best Practices for Regression Testing
Now that we have discussed the nuances of regression testing, let us look at some best practices that one must follow.
- Implement a regression framework from the beginning of the project, with detailed test designs to ensure that no rework is needed at a future stage.
- Regularly reassess developers’ and users’ needs and update test designs accordingly
- Analyze the user experience on your web application to narrow down highly used functionalities to focus test designs on
- Add new tests to the testing pack when new functionality is added to the application
- Regression tests are almost always automated, and therefore testers must be experienced in writing test scripts for a variety of test cases. For example, regression testing can be done through Selenium and BrowserStack, where the method of defining the test remains the same – but the process of execution is through a browser on a real device. BrowserStack can thus facilitate quick, convenient automated testing on both desktop and mobile devices.
For more on how to speed up your regression testing, check out our post on how to speed up regression testing.
While regression testing may require some additional resources, it ensures significant benefits that justify the investment, both in terms of time and cost. It is an integral part of an agile development cycle that focuses on maximizing output and promises to make life easier for developers and QAs while also providing a good user experience.