Test Automation Effort Estimation

Software testing ensures your product meets all customer requirements and addresses each product risk with a proper test plan. Testing can uncover failures of many kinds, and in today’s agile world, automation testing is a part of the work to be done for the story and the benefit of test automation is quite obvious. A properly planned testing process is a must for ensuring the required level of software quality without exceeding a project’s time and budget.

What is Effort Estimation and why is important

Testing is no longer a one-time task but a continuous process. An estimate is a prediction, it could be an approximation of what it would cost, or a rough idea of how long a task would take to complete. Better testing estimates allow for more accurate scheduling, realize results more confidently, and form the foundation of good management credibility.

What are the factors affecting test automation estimation in agile

Estimating testing in the contemporary world of agile and DevOps demands some new rules. Stories with dozens of tasks that lack the value definition for intended users is not helpful. One need to consider the effort required for test design, test implementation, test execution, and the effort to automate while estimating the story. Especially when you are conducting an automation testing to reduce the cost and time, you should consider the below estimation techniques.

Scope

Scope usually means finding the right test cases for automation. The deciding factor for any test to be automated is linked with how many times that particular test can be repeated. When new functionalities get added or if there exist any modification in existing functionality then the automation scripts need to be added/ modified, reviewed and maintained for each release cycle.

• Split the application into separate modules and analyse each module to determine possible testing options

• Prioritize the list of modules and assess the history of each and every test case (as many of the test cases may not have been run often enough to get into the right priority)

• Analyse and classify the Test cases into various categories like API, Web, DB etc and choose the ideal candidate for automation

• Identify the complexity of the test case like Number of verification point or check points for each of the test cases

• Calculate the ROI for the identified test cases

Supporting Tools

Selecting an automated testing tool is essential for test automation. There are plenty of automated testing tools available in the market, it is important to choose the automated testing tool that best suits your overall requirement. Consider the below factors when selecting the automation tools

• Understand your project requirements and identify the scenarios you want to automate.

• Finalize the budget for automation and search the tool that suits your requirement.

• Compare and choose the tools that fall well within your budget and easy utilisation.

• Check for Logging and Reporting mechanism

• Underlying language which is used to build automation script and check for the feasibility of integrating with the build tools

Below is the combination of tools selection for framework development:

1. Eclipse / RAD – as IDE

2. Ant / Maven – As build tool

3. jUnit / TestNG – as unit test framework

4. Log4j – as Logger

5. ReportiNG – as reporting tool

6. Text files – for tracking the environments/credentials

7. Excel files – for tracking the test data

8. Perl / Python – for setting up an environment and triggering the test scripts.

Framework Implementation

Framework implementation means designing input files like Excel / JSON / XML etc, designing folder structure, implementation of logger, utilizing build tools and environment related data and credentials.

Framework means a real or conceptual structure created to provide support or guidance to an entity that could expand in future. In simple terms, a test automation framework can be defined as a set of processes, standards and interactions between the components in which scripts are designed and executed. While implementing a framework one should consider handling scripts and data separately, reuse of code, high extensibility, library creation and maintainability.

We are detailing out the below variables and calculation methods that helps in estimation:

Test Case ComplexityWe classify the Test case based on the complexity of the Test case and the complexity of the test case is based on four elements including checkpoint, precondition, test data, and types of test case, which effectively assumes that the complexity is centred at these four elements. Below is the complexity assignment of the test case.

Transactions are measured while hitting /submitting a page / screen. In an ideal scenario a simple Test case should have less than 2 submits or it covers less than 2 navigations from a page. When we read the Test case, we focus on these navigations to calculate the Test case complexity.

Interface complexity

Estimators view and review the user interfaces, API, DB and network to measure the accuracy of the interfaces in providing the service. This complexity describes the external integration/interfaces that are associated with the proposed software. These integrations could be hardware, software, network and communication based on the deployment of the proposed software.

Verification Points

Verification Points are used to verify whether the value of an Application Under Test matches with the expected value. This output value can be considered as input data for other verification points. The complexity is measured using the Test Conditions, Test Data identification, Test Script preparation and Test Execution.

Listed below are the other variables which we need to consider while estimating

While it is quite obvious that the strategic goal of automation testing is to ensure business value, it is just as essential to articulate it periodically to stakeholders on where we stand in the journey towards it. This can be projected in two broad areas – savings in terms of time and cost. Our estimation approach must be dynamic and responsive to changing requirements with the various variables and components described above and the effort is calculated based on test design, test implementation, test execution, and the effort to automate. I hope this will give a head start to those who are seriously considering to calculate the automation testing effort estimation.