Regression testing is undeniably important in software testing. We need to regress the software product as and when, we add new feature to our software product or modify our existing software feature or after bug fixes or enhancement user story.
I recollect from my initial days, we execute around 400-500 test cases during our regression test cycle. Regression cycles were lengthy and exhaustive and testers tended to lose focus.
Regression testing is always an integral part of any software development life cycle whether it is Waterfall, ADHOC, V model or Agile. By doing a Regression testing, we are confident over the software product quality.
Organizations invest huge amounts of efforts, resources and money on regression testing and automating regression test suites.
If we plan, design and strategies our testing well before, we can definitely achieve less painful and better coverage in regression testing during sprints as well as in final release.
Definition of regression testing:
Regression testing is performed to determine whether a changed component has introduced any error in the unchanged component.
During the software development process, bugs are likely introduced while implementation of change in existing business functions, introducing new features, enhancing existing features or during bug fixes.
We can perform regression testing by selecting correct test cases. This can be achieved by understanding impacted modules, understanding priority of the test cases and grasp of impact prone modules.
As we develop user stories in each sprint, their test cases pileup in the regression test suite. As each iteration increments, the number of test cases considered for regression testing increases.
A case study on the banking domain shows that regression techniques and testing, helped in saving much as 60% of the time spent on fixing bugs as 40% in terms of the money eventually spent use of regression without the use of regression testing.
Regression Testing strategy in sprint:
As a manifesto of Agile development, continuous communication is the key.
- Whenever a new user story is developed or a support bug is fixed in the software product, we need to consider regression testing scenarios.
- We can reduce our burden just by communicating well and communicating in advance with the scrum team. Close communication with developer on changes carried out the code during new feature or understanding impacted modules will give more insights while selecting test case for retesting
- Communicating with developer to understand impacted module, analyze the risk and mitigating the risk by adding the test cases as regression of the user story or bug
- Communicating with Product owners and analysing and understanding the priority of the impacted features. So that important business functions will be retested thoroughly.
- We need to be mindful and attentive of our previous experiences. Considering impact prone modules or functions need to be considered while designing and selecting regression test cases.
*Previous experiences- That means vulnerable modules breaking frequently or getting impacted by updating in SQL procedures or updating table views or code changes.
Techniques to select test cases for Regression Testing:
An agile methodology has the incremental, iterative nature of the software development process that requires a huge amount of effort spent on regression testing every iteration, as new features and stories are added to the system much more quickly.
Selecting Test cases for Regression Testing:
- Retest : When major changes are done on code. Then we need to consider retesting that business function.
- Selective regression : Test cases from likely affected functions or fail prone modules can be selected.
- Prioritizing : Understanding selected test cases has business impact and prioritizing them. Selecting the test cases such that important business feature get covered
- Every time a story is completed, not only it must be tested, but all previously completed stories regress with some degree of retesting.
- Regression test scripts are updated and executed in the sprint when necessary. This is part of the User Story and should be taken into account when estimating.
- Regression Tests has main subset Sanity testing and retesting. Considering these subset while test case selection will help in optimizing the regression test suite.
Sanity testing is performed to test the core behaviour of the application. It normally includes a set of core tests of basic GUI functionality to demonstrate the connectivity to the database, or application server or external application, third party APIs, Printers etc
Retesting is rerunning all the test cases/scripts.
Conclusions:
Regression testing has an important role in Agile development. In addition to its usual purpose of providing stability, it helps the team focus on the functionality in the current sprint.
Attempting to execute functional tests and regression tests together on each user story, bug fixes will reduce the load in the regression cycle performed at the end of release.
It will also help in finding the regression bugs in same user story which is implemented in that sprint. Thus, bug slippage will reduce and eventually cost will be reduced.
These are some strategies, test designs and planning helps test team in performing regression in sprint.
Credits:
Blake Norrish : https://medium.com/slalom-build/the-regression-death-spiral-18f88b9fb030#_=_
Ajay Sarangam : https://www.jigsawacademy.com/blogs/data-science/regression-testing for statistics on regression testing
Heartfelt thanks to my teammates and my managers for giving me exposure, inspiring to do better testing.