First of all let us see the meaning of the term Regression Testing

According to Wikipedia, "Regression testing is any type of software testing which seeks to uncover software regressions". Such regressions occur whenever software functionality that was previously working correctly, stops working as intended. Typically regressions occur as an unintended consequence of program changes. Common methods of regression testing include re-running previously run tests and checking whether previously fixed faults have re-emerged.

Concluding Definition of "Regression Testing":

It is quite natural to have changes in the code during software development and maintenance phase. Thus a software program that happened to execute properly earlier might stop functioning the same way after the changes. Hence re-running of the test cases of a program, which functioned correctly before the changes, and with an objective of detecting failures, can be concluded to be "Regression Testing".

Regression testing is a quality control measure aimed at ensuring the following two conditions:

a) Newly modified code meets the specified requirements.

b) Unmodified code has not been affected by the change as above.

Going by the definition, regression testing is repetitive. Hence majority of the tests would be best suited to automation, where after performing few test iterations, test automation would prove to be extensively costs effective as compared to the strenuous manual process.

Why defects get introduced during changes?

Correctly working applications fail due to either incorrect or incomplete changes made in them. In software industry, the rate of defect introduction in the applications is quite high. In general, one out of six attempts made over the applications to correct them are faulty themselves.

The prime reason of high rate of defects getting introduced is

1. Poor system documentation held by the developers.

2. Tendency of the developers to tackle the symptoms instead of identifying the underlying causes.

3. Lack of experience of the developers.

Objectives of regression testing:

1. The objective of regression testing is to identify unexpected defects. These defects or faults are the ones, which remained in the system, due to the reason that, while changing the code, the developer probably could not completely understand the internal correlations of the code.

Hence regression testing remains the only reliable means to provide adequate confidence that changes or additions in the code are safe & are not liable to "break" the existing functionality of the application.

Regression testing thus becomes a must, every time code is modified or is used in a new environment. It is essential to verify the integrity of the code with a view to identify & fix the defects as quickly as possible.

2. Objective of regression testing is not only limited to testing the correctness of an application but extends to track the quality of its output as well. For example, while designing a compiler, regression-testing can lay focus on the size of the code, time for doing simulation and time to run the test suites.

What are the Best Practices / strategies for Regression Testing?

1. Formulate a policy for regression testing on regular basis, if we want to achieve success in developing reliable & definitely predictable software applications.

2. Perform standard actions defined in the testing procedure & check the desired responses for correctness. Any failure of the system to comply with the set of desired responses becomes a clear indicator of system regression.

3. Ensure that the regression test must be correct & should not be out of date.

4. Do careful analysis of every defect escaping detection during the process of regression testing. This would need a careful updating of the regression test cases, so that such defects don’t give us a slip in future.

5. Generally unit test cases & integration test cases are used to build regression test cases; hence instead of having one large regression test, it is better to create a logical batch of such test cases in the form of a comprehensive test suite. This would provide flexibility of executing small unit tests in case of urgency or time-pressures.

6. Going by the famous 80 / 20 principle of management, we can assume that there is a great probability of 20% functions being used during 80% of the time. Hence test suites for regression testing can be designed accordingly.

Severity of a problem in a comparatively lesser common functions would be lesser being used by small number of users. Thus major focus of our regression tests must lie over excessively used transactions, menus & screens etc.

7. For smaller projects, do regression testing after every successful compile or at least once in a week. Such repetitive activities can be automated with the help of tools like HP-QuickTest Professional etc.

8. Depending upon the risk factors across the business, we can design the regression test suite. It has been seen that there are certain types of failures, which are not frequent, but whenever they happen, they leave serious impact on the business process.

Hence whenever any change is made to the system or to the environment, we should perform regression tests covering such vulnerable areas having negative impact on the business.

9. We need to identify the application areas known to have high rate of failures & include more regression tests in them.

10. Tightly link the regression testing with functional testing & create the regression tests out of successful test cases created & used during the functional testing.

11. Regular rerunning of successful functional test cases (which happen to verify the desired functionality of the application) as regression tests.

12. Regression testing should be like a chain, starting from the unit level, involving adaptation & rerunning of the unit tests after sensing any change to the unit. The chain of regression tests continues further across integration testing, system testing, user acceptance testing and across all operational phases under the SDLC.

13. As a best practice, before releasing any build across the masses or across any live environment in the organization, stringent regression tests must be executed. Such an approach would help in uncovering major defects if any, which could otherwise have serious Implications towards cost, productivity, schedule & most important being the adverse effect on company’s reputation.

14. As a best practice, execution of regression tests at regular intervals should be made a continuous exercise in case of Web based & all multi user systems. Regular regression testing helps in maintaining a continuous check on the performance of important transactions being within the specified limits.

Any factor responsible for slowing down the response time for any major transaction can be easily pin pointed by frequent regression testing.

15. Apart from testing the functional aspects, we need to perform regression tests covering non-functional attributes of the application like its performance, security, usability etc. Reason being, it has been seen that in many cases, a minor change in the code or design produces considerable effect on the system performance.

16. Before running the time consuming regression testing scenarios over newly delivered code, doing Smoke Testing or Sanity Testing is helpful in saving time, since Smoke Testing or Sanity Testing generally uncover the obvious errors. Early detection of such errors can lead to early fixation.

17. We should have a careful eye on the side effects of the bug fixes. There is great probability that the bug gets fixed but at the same time our fix could have introduced another bug in the system.

18. Regression testing must be treated as an integral part of the extreme programming method of software development. This would involve extensive automated testing of the complete application at every stage during the SDLC.

Pitfalls of Results of Regression Testing

1. We should not become happy & get satisfied just because the system has responded as desired, since there can be every probability of presence of introduced defects, which might have escaped the detection.

2. In majority of companies the critical functionality of the applications are verified once, and an assumption is drawn that it would continue to behave well until modified intentionally. However, the fact remains that even minor changes in code as a matter of routine are liable to have serious unexpected side effects, which can probably break the functionality verified earlier.

What are the Benefits of having a Good Regression Testing Policies?

1. Great improvement in effectiveness of the software development & testing personnel.

2. Great success of the development projects resulting in reliable & stable applications.

3. Development teams modify the code without fear of breaking the previously verified functionality.

4. Problems arising out of code modifications get identified early during the life cycle.

5. Great saving of man-hours spent in hunting for & resolving software faults introduced by code changes.

What are the attributes of a Good Regression Testing Policy?

1. It defines sound guidelines for regression system usage.

2. It defines concrete plans to consistently implement & integrate the defined guidelines into the SDLC. This results in detecting existing errors as well as the newly introduced errors due to unchecked modifications made in the code over a passage of time.

3. It provides mechanism to measure & monitor the application of the policy and system to report the data being tracked.

4. Sound documented system of controlling & maintaining all types of regression test suites being company assets of great importance. The policy outlines system for backing up, configuration management, maintaining the latest ones & defining clear-cut ownership and responsibility of specific personnel to manage the regression test suites.

What are the reasons that Regression Testing is not so popular?

Despite many benefits of regression testing, it is not practiced by many organizations probably due to the following reasons.

1. Software development companies feel that regression testing is a bit complicated and difficult to maintain.

2. In most of the cases, companies don’t have some well-defined & implemented policy towards regression testing.

3. Even if some policy is available, generally there is a lack of commitment from the management side towards implementation of such policy.