Regression on a Daily Basis

The client was on the lookout for an automation outsourcing company for their regression testing as they had an in-house team that took care of the manual testing needs. Their need was very unique as they needed us to perform regression testing on a daily basis.

The reason behind such a need was that they had to update their builds every single day. And this meant that they either had to perform a nearly impossible task of manually testing all of their 10 modules which would take one whole week as they had to cover 10,000 test data per day.

Our Solutions

• 1-week’s worth of tedious manual testing was initially reduced to just 15 hours using automated regression testing.
• We boosted their ROI by further reducing the execution time to just 7 hours by overcoming challenges that were unique to the application.
• We have automated a groundbreaking number of 600+ scenarios.
• Since these regression suites ran every day of the week, we provided round-the-clock 24×7 support.
• We even delivered consolidated reports on a daily basis and performed uninterrupted script maintenance.

Our Heavy Duty Regression Test Suite

We were instantly excited about the project despite the daunting challenges as we were 100% confident in our abilities. That 100% confidence has resulted in us being able to automate 600 unique bug scenarios from scratch. But it was no easy task as we had to execute automation scripts for 15 hours every day on a virtual machine during the early days. The tireless effort of our test automation engineers made it possible to flawlessly perform such an enormous task day-in and day-out for the entire first year.

Going the Extra Mile

Due to the unique nature of the application, conventional parallel execution was not possible as it had various conflicting scenarios that would cause many failures. So once we had earned the client’s trust, we were able to propose a much more efficient testing process that would reduce execution time by half. We suggested our client to create & use 10 instances of the same testing environment to run to overcome the conflicting scenarios problem. Though it demanded more infrastructural needs, our client understood the potential and increased our Virtual Machine’s capacity. This enabled us to launch 10 browser instances at once and carry out different tests in the independent test environment instances without any conflicts or failures. We have now reduced the execution time to just 7 hours.

In addition to the test execution, we also performed audits on the failed tests to see if there were any issues in the automation script by manually testing it. Based on the results, we were able to identify if there was a problem with the script or if the failure was caused due to a bug. The long execution times meant that there was no room for errors as we had to file our daily reports.

Future Plans

Being an ongoing project, we are proud to say that we have not received even one escalation over the past 3 years despite it being a very demanding project. In fact, we have made additional contributions that go beyond our SLA by streamlining the testing process, identifying previously undiscovered bugs, and reporting them to their in-house QA team.