Regression testing is conducted to ensure, the code fixes have not caused any new breakage of the application in some other place. In this test instead of running the failed case alone, we run all the test cases that were passed before to confirm the presence or absence of defects. Regression testing helps us to find the stability of the applications as we pass by the timelines.
This test should be run with an agreed frequency to ensure that addition of new functionality, integration of any other new component and any bug fixes have not affected any other existing system functionalities.
Why is regression testing necessary?
We might have a question in mind that why to spend time in testing the same again and again though it was passed before. But there could be a high chance that new thing will inject a new problem which didn't exist before.
Below are the possible scenarios as to when it’s highly recommended.
When we are adding up new features
In any application development projects it’s true that as long as we pass by different sprint cycles we keep adding the new set of features to the application. Every time there is a new feature developed on top of it regression testing must be done in order to ensure that the new flow or use case is not impacting other existing functional flows.
When bug fixes were delivered
When the development team fixes the bugs observed in the system it’s the responsibility of the test team to consider them for re-testing then do a regression testing. Let us quickly understand the difference between both of them.
1. Re-testing- fixing the bug that was fixed to confirm it’s no longer happening.
2. Regression- ensuring that fix has not affected any other related functionality.
So, now we understood the importance of doing both the tests. This testing can help us realize which broke the system that was doing fine earlier. Based on the newly occurred bug impact as to how it is affecting or blocking the testing activities accordingly it would be rolled back if needed.
When application re-platforming happens
Often times we see that the applications are considered to be re-platformed. Especially after the cloud technology emerged this has started happening a lot, many applications are considered for migration for the conventional virtual environment to a cloud infrastructure.
Usually in the migration activities we don’t see any new changes to the application at the functional level. Whatever the existing system would just be moved or re-hosted. In this case, regression testing is mandatory because the whole application was migrated and need to confirm that there it works intended.
Types of regression tests
As regression appears to be a redundant test, since we do the same thing, again and again, irrespective of the previous outcome, whether or not they pass just to learn new issues. It also becomes overhead if we don’t have a proper regression plan. Because there would be many challenges with regards to timelines, resources and budget and a lot of other constraints to contribute. Given those parameters, there can be two types of regression tests.
Complete regression test
Full regression test is needed on the following cases
When there are significant changes into the application from the functional/non-functional standpoint.
When we have major launch release is planned.
When the core functionalities or the business logic were modified.
In order to be able to test thoroughly regression suite has to be updated every now and then to ensure we keep the suite of the latest one. Through this testing, we will understand that the application didn’t break and is working intended.
Partial regression test
This kind of testing is mainly adopted when we have changes to a particular module within the application and the timelines for testing are very limited or a small release with minimal bug fixes is planned. All we need to satisfy the case is just identify the edge cases and the scenarios where the integration of this module with other modules happens. These cases should be identified based on the experience also from the past learnings.
Given this rationale sanity test is called a subset of regression test as we will not execute the whole test cases given the time crunch. We will pick scenarios of the critical edge case.
Automation testing role in regression
If there is a type of test where automation can best fit is regression. Automation implementation in regression can give the best return on investment as the manual effort that is being spent is reduced to a great value. In addition to that automation testing also brings inaccuracy. The testing team needs to leverage the suite scope by adding the features delivered in each iteration. After every iteration running one set of regression execution can help to find the possible side effects.
DevOps integration with regression test suite
As we all know that DevOps is the buzzing term in the industry at this point in time with the help of continuous integration and deployment now the team can achieve frequent builds and frequent deployments we can avoid the possible downtimes. The same can be implemented in testing as well, integration of regression with DevOps will help us to manage frequent executions. As the execution frequency is more the intensity of the regression test becomes more. That’s what leads to the successful validation of the software product. The more we do regression less likely the system fails in production.
Regression Testing Best Practice:
Identify incremental Regression Test cases based on releases.
Plan effort dedicated for performing Regression Testing (Thumb rule is 30% of Functional Testing effort should be spent in Regression) for every release.
Accommodate a percentage of Regression comprising of ">End to End Testing as well.
Have a session with Business team to identify critical scenario based on Application functionality and accommodate it in the Regression suite.
Plan to have maximum Regression coverage in Automation to minimize Test Execution effort.
Major defect fixes, each cycle completion of Test Execution, Major deployment are all qualifiers for performing Regression Testing.