First, the basics. The term "regression" is used to describe the decay, weakening or degeneration of software. Software is modified to add enhancements to it, fix known defects, make performance improvements, make the code more maintainable or make the code compliant to patterns. But, software can regress when it is modified. This can happen in a number of ways:

a. The faulty design is re-used to create an enhancement

b. Bug fixes get rolled back

c. Prior bug fixes do not work anymore due to changes in the environment (e.g. a new browser version)

d. A bug fix in one part of the software creates a new bug in the same area or a related area

View my Regression Testing tutorial or read on... 

Regression Testing

Regression testing determines if extra defects have been created in the modified software. As a tester, you are faced with a number of choices while performing regression testing:
1. At what levels should you perform your regression tests?

2. Which test cases do you execute (the existing test cases or new test cases or a combination)?

3. How frequently should you run the regression test?

4. Which part of the regression test should be automated and which part should be manually executed?

5. How do you ensure that your regression tests are effective?

6. How do you ensure that your regression tests are optimized?

You can come up with the best way to do regression tests if you select the best options to the above questions. Therefore, let us discuss these questions in detail.

1. At what levels should you perform your regression tests?

As you know, software testing happens at a number of levels (e.g. unit testing, integration testing, system testing and so on). Since the focus of testing is different at different levels, you are going to have a better likelihood of finding regression defects if you perform regression testing at different levels. For example, you may find defects in unit testing which would not have been captured by your existing system tests. You may find defects in integration testing which would not have been captured by any unit tests.

2. Which test cases do you execute (the existing test cases or new test cases or a combination)?

First of all, you should be aware of the impact of the changes to the software (since the last regression test). Then select a suitable set of test cases that sufficiently exercise the impacted areas of the software.

You may want to use the existing test cases if they are sufficient in their coverage. Further, they should have been updated along with the prior changes to the software e.g. the test cases are updated based on the defects discovered in the past.

If you know that the existing test cases have skimpy coverage or out of date, you may want to create new test cases for your regression test.

You should also ensure that there are no duplicate tests in your chosen regression test suite.

3. How frequently should you run the regression test?

This depends on the cost you incur in running your regression test and the value you receive out of it. Even if you don't run a regression test every day (note that some teams do), you should at least run one complete regression test before the software is released to production or to your client.

4. Which part of the regression test should be automated and which part should be manually executed?

Tests which are stable, repeated frequently, simple and require no intervention by the tester are good candidates for automation.

5. How do you ensure that your regression tests are effective?

Your regression tests should be able to discover defects. Upstream regression tests should discover a greater number or the more severe defects or at least discover defects more easily than downstream tests.

6. How do you ensure that your regression tests are optimized?

If you are aware of the scope and timing of the build process, you can align your regression tests with it. This will lead to an optimal number of regression test runs.

Further, you should examine your regression test cases to eliminate duplicate test cases, merge test cases wherever possible and automate tests (based on the criteria above) to minimize the time/ effort it takes to run your regression test.

Want to learn more? View the video that explains Regression Testing.