After modifications have been done to an existing application, you need confidence that those modifications have not negatively impacted other functionality. That’s where regression testing comes in. While absolutely necessary, regression testing can create a bottleneck in the SDLC, especially if it’s not automated. In addition, regression testing alone can encompass up to three-quarters of the total cost of testing.

A good regression strategy is necessary for timely, cost-effective, and thorough test execution. Running the entire suite of tests that could number in the many thousands is prohibitively expensive and time consuming. Execution scope must be decreased in order to reduce the expense and time frame of a regression. There are different techniques for regression test selection, but we have developed our own methodology at QA Mentor that helps reduce costs and decreases the time spent testing, while still offering good coverage.

Our first step is to meet with the development team to assess all of the changes made to the application and their impact to the existing functionality. This impact analysis is best done with the team that coded all of the changes since they are the ones who know exactly what they wrote and how it integrates with other parts of the system. During this meeting we work with the development team to isolate modules or functional areas that were not affected by the changes in any way. Once we have eliminated the functional areas that do not need regression testing, we can focus on identifying the ones that do, and to what extent they require additional testing.

Next, we tackle the regression test repository. We identify tests that are now obsolete due to current or past changes. We also modify existing tests to reflect the changes made to the system, and create new ones that may be necessary due to added functionality. It’s very important that the test case repository is properly maintained in this way at every release, as it speeds up future testing and makes it more accurate and effective.

Once the regression suite has been updated, we identify and select test cases for each of our three phases of regression testing. When selecting test cases for any regression phase, we take a risk-based approach to select the appropriate test cases and avoid multiple negative scenarios. The risk analysis is helped in part by the initial conversation with the development team.

The first stage of regression testing we select test cases for is a Focused Regression. It covers – or focuses on – the specific functionality around the new or modified code. The regression testing of affected areas is done in tandem with the functional testing of the new or modified pieces. Some testers on the team focus only on areas impacted by the changes, and others work solely on the new or modified functionality. In this way, regression testing isn’t done at the end of the cycle and instead started closer to the beginning so that defects can be found and fixed earlier. Experience has taught us that the new or modified areas sometimes do not work well with other sections of code, forcing the entire modification to be altered or even new modifications added. It’s far more efficient to discover this earlier rather than later.

Maintaining Regression Test Inventory

Once we have identified the focused regression test cases, we select the test cases for the Expanded Regression, which incorporates tests for functional areas not directly related to the changes, but possibly affected by them. This regression phase will be executed after all coding is complete and the code is frozen. If possible, it’s done using an automation tool but most often the test cases are manually executed.

Our next set of tests selected from the repository are for the New Release Functional Regression Testing. We select 30% of the newly added functional test cases as a mini-regression at the end of the cycle. This phase focuses only on the new enhancements, not existing functionality. Due to the earlier testing where defects were found and fixed, this final regression is performed as an extra safety precaution. Code changes were made in the previous regression phases, possibly multiple times. This final step is aimed at finding defects that may have been overlooked during the find and fix periods at the beginning. It also offers the opportunity of a higher level of confidence in the final product. There’s no need to execute all of the tests again, but we’ve found a 30% sample is more than sufficient.

Once we have all of the tests selected and grouped for the three regression phases, we establish an execution path for each phase and begin testing as soon as code is ready. With this methodology, we know we can reduce regression testing costs by reducing the time it takes to test, and without sacrificing quality as a result.

Awards & Recognitions in Software Testing Services