Blog

Test Automation Challenges and Strategies

by admin | Feb 23, 2020 | Automation Testing, Fixed, Blog | 0 comments

Nowadays most companies invest a huge amount of money in Automation testing, but the success rate in test automation is very less (25%) due to various reasons. In this blog, we are going to discuss the most common challenges that the Organization faces in automation common challenges that the Organization faces in automation and the best strategies which when implemented will yield better results.

Below is the list of challenges that are widely faced in Automation:

• Lack of Continuous Monitoring
• Management / Stakeholder guidance
• Lack of Knowledge in Functional Testing team
• Lack of Understanding of What to Automate
• Automation team skillset and Training
• Attrition Rate and Ability to retain good resources
• Lack of Team bonding and co-operation
• Lack of Ownership
• Right Tool Strategy
• Right Framework standards
• Lack of Development practice with good code review process
• Quantity conscious rather than Quality conscious
• Lack of Governance

Lack of Continuous Monitoring

Automation should be treated as a development practice. Like how development is monitored on a continuous basis, Automation Testing progress should also be monitored and closely watched. Below are the continuous monitoring activities that every development work is governed and the same should be applicable for automation as well.

• Reviewing the quality of work
• Validating the delivery accomplishment as per the agreed plan
• Evaluating unforeseen risk based on the outcome
• What are the activities that need changes based on priority changes from time to time
• Expected Result Vs Actual Result assessment
• Statistics of project progress

These activities are applicable for Automation Testing as well and this needs to be assessed and evaluated time to time.

Management and Stakeholder guidance

Time to time Management and Stakeholder review should be conducted and guidance to be provided to achieve success. Below should be the key people in such people

• Automation Lead
• Functional Testing Lead
• Program sponsor
• Development Lead and Manager
• Testing team Manager
• Automation Architect if such a role exists

Such a meeting should be conducted on a scheduled frequency and discuss on achievements, hurdles, and solution to hurdles.

Lack of Knowledge in Functional Testing team

30% of Automation failure is due to the lack of knowledge on Functional aspects from the Functional Testing team. The functional team should be in a position to guide what to automate and what not to automate. The functional team should know the challenges in automation and help the automation team with the right area of Automation. Below is the expectation from the Functional Testing team to get good Automation result.

Identify the right area to Automate

1) Instead of automating UI, there could be an API that carries the data to UI and API can be automated instead of UI. API Automation is easy and simple. The focus should be given on such areas

2) If Functional team does not have an understanding before taking up Test Automation activity, we should discuss with key stakeholders and come up with such areas

Test Case Revamping

When there are at-least 100+ Test cases to automate, Functional team should come up with scenarios which can club multiple Test cases without compromising coverage. This will reduce the effort drastically on Automation as well increase the quality

Test Data Management

1) For automating any test case, test data is pre-requisite. Functional team should add such requirement as a part of Test case preparation activity

2) Should also have a clear idea of how to generate the data before taking up automation work.

Basic Feasibility

Based on the nature of application and dynamics involved in it, the functional team can suggest what application to be automated and describe the possibility of Automation

Lack of Understanding of What to Automate

Many Organization focuses on automation based on funding allocation rather than based on the understanding of prioritization. This leads to poor automation with less ROI.

Since time is not invested in what to automate, the focus goes towards the quantity rather than quality.

Test Automation Team Skillset and Training

As a test automation services company, we always encourage our automation testers to learn continuously. Automation team needs to learn new technologies, new methodologies and better coding standards year on year. This needs frequent workshops and frequent reviews to improve ourselves. Also, we need to enable the existing functional team to move to automation to reduce the attrition rate. Hence skill improvement and training should be a continuous process and the team which does these activities excel in their outcome.

Attrition Rate and Ability to retain good resources

Retaining key players is a big challenge in any industry and in the automation area, it is inevitable to take measures to retain resources. Below are the few areas to reduce the attrition rate.

• Rewards and recognition to star performer.
• Identifying the right resource for the right skill
• Job rotation if the same job can be done by multiple resources
• Giving Exposure to the team to key stakeholders
• Monetary benefits
• Giving space for recreation and fun activities to improve the moral

Lack of Team Bonding and Co-operation

Many occasions due to lack of team bonding, a lot of confusion and lack of ownership happen. We need to identify skillset and interest levels to allocate the activity. Team building exercises have to be conducted to create better bonding

Right Tool Strategy

Based on the Organization policy, fund allocation and based on Application technology automation testing tool procurement will be decided. Before fixing up tools, below are the areas to be considered

• Tool feasibility across Technology platform has to be conducted before making the tool decision
• The capability of the tool beyond automation (Like integration with other tools), plug and play support, etc to be assessed
• Resource availability with the proposed tool skill in the market and within Organization
• Support for any technical help through forums or through tool vendor to be assessed
• Tool vendor support in procurement process like a waiting period for evaluation time, support from their team on technical feasibility, training to the Organization
• Rate each of the tools and their vendors based on various criteria’s and come to the right tool strategy for the Organization.

Right Framework Standards

Before doing automation across applications, form a team for Organization Global framework development and Maintenance which will help to create dedicated focus on this activity. When the Organization matures, this framework will also mature and provide good ROI during DevOps implementation .

This dedicated team can assess different Technology stacks involved and create Framework based on Technologies.

Since the delivery and Framework team is separated, there will be a seamless delivery and continuous improvement.

Lack of Development practice with a good code review process

Automation work should be treated as a development practice. Usage of tools for Configuration Management, Automated code inspection tools, Build Management, Continuous Execution is highly recommended.

This also can help in early detection of code quality and issues and fix from time to time. This can also help to easily identify the weak resources and provide adequate training

Quantity conscious rather than Quality conscious

Management should be quality conscious rather than quantity conscious. Record usage of existing scripts Month on Month to increase ROI. Keep an eye on what improvement we have brought in and what we have done differently across each application each month to improve quality. Always upgrade the Lessons learned from each activity for implementing better strategies.

Lack of Governance

During governance discussion, numbers/metrics are tweaked to meet the expectation and hence there is less focus on improvement.

During governance, constructive discussion on the challenges faced, a solution adopted and other opportunities we have to improve and implement can be discussed rather than focusing on just Metrics.

Strategies for Test Automation:

• A proper assessment on application and technology stack
• Right tool strategy by doing a proper feasibility study<
• Right framework approach keeping in mind of scalability and adaptability
• Retaining skilled resources
• Conducting training and upskilling resources
• Infrastructure and Environment for Automation
• Identifying the right area of Automation rather focusing only on UI
• Constructive Governance rather than pure metrics-driven
• Stakeholder Involvement

Learn How QA Automation ROI is Calculated

by admin | Jan 28, 2020 | Automation Testing, Fixed, Blog | 0 comments

Are you curious as to how businesses calculate ROI for automation testing? Since test automation is expensive, the decision of whether to invest in automated testing tools or continue with manual testing can be challenging to make. Let’s break it down for you and simplify the various ROI calculation formulas on test automation and help you make an educated decision.

Usually, the primary type of calculation is a simple ROI calculation, but in this blog, we will delve into advanced calculation techniques.

Efficiency ROI calculationIt needs to be done in terms of days since automated tests can be run continuously as compared to the 8 hours calculated for manual testing. We calculate this with an average of 18 hours as test cases are usually interrupted and do not always run for 24 hours. The formula, then for efficiency ROI calculation on test automation is:

Key: Automated Test = AT, Execution Time = ET, Maintenance Time = MT, Manual Test = MaT, Period = P

• AT script development time = (Hourly automation time per test * Number of AT cases)/8
• AT script ET = (AT ET per test * Number of AT cases*P of ROI)/18
• AT analysis time = (Test analysis time * P of ROI)/8
• AT MT = (MT * P of ROI)/8
• MaT ET = (MaT ET * No. of MaT cases * P of ROI)/8

Notes: The period of ROI = number of weeks for which ROI needs to be calculated. Divided by 8 for manual automation and divided by 18 for system automation.

To calculate efficiency ROI, rather than monetary factor, we consider time investment profits. We input the values of the respective variables in the formula and arrive at the answer. Although you are not required to disclose the cost to the company for hiring a tester or other such sensitive information, this method makes plenty of assumptions instead. Even though you automate test cases, you may still need manual intervention at times.

Risk Reduction ROI

In this ROI calculation method, automation benefits are calculated separately while addressing concerns that arose from the approach mentioned above. When you adopt automated testing, resources become more available to do other productive tasks like in-depth time analysis, independent negative/random testing, design/development of the tests, etc. It increases coverage, and you can discover bugs in your application and fix post-delivery errors. If any bugs are overlooked, then the consequent loss needs to be factored into this ROI formula. Any gain in terms of money, if errors post-delivery or implementation are found and fixed, will directly impact the calculation. Values are inserted in the same formulas, and then you can calculate the ROI. Even though the investment cost is similar, the biggest gain is the monetary loss that the company would save itself from if automation testing is implemented. The advantage here is that it deals with the positive effect of test coverage, a high level of risk analysis because this method is subjective.

Several factors affect ROI calculation for automation testing, so you should be advised by a reputable ?automation testing services company?. Add efficient testing automation tools, invest more time in the initial phase, and use better reporting mechanisms to introduce improvements to your automation testing. There is no singular method to calculate ROI for automation testing, so it is best to hire a t?est automation services company? like Codoid. A consolidated decision between your testers and management will lead to the best automation testing method installed in your business that is sure to bring forth profits and help you achieve your goals quicker.

Best Practices for Automation Testing with BDD

by admin | Jan 23, 2020 | Automation Testing, Fixed, Blog | 0 comments

If you follow automation testing best practices religiously it will eventually decrease rework. Nowadays, BDD frameworks have strong automation testing user base. BDD framework enables effective collaboration and automation. If your team is following Agile methodology, then make sure you automate Acceptance Criteria of each story within the sprint. The reason being if any automation test script is being developed with effective collaboration, it will produce high-quality output. Let’s say you have missed to automate a couple of stories in the current sprint, then the non-automated acceptance criteria becomes a technical debt and you may face difficulties to connect with your team members to discuss about old sprint stories.

If you are automating existing regression tests pack, you can use BDD framework only for automation without collaboration. Writing BDD test scenarios is an art. None of your team can understand if a Gherkin scenario is too long. Following best practices is an essential for successful automation testing with BDD.

Feature Template

Feature: [One line describing the story]
[Optional — Feature description]

Scenario: [One line describing the scenario]
Given [context]
And [some more context]…
When [event]
Then [outcome]
And [another outcome]…
  

BDD Scenario Best Practices

• Given-When-Then should be in sequence.
• Each scenario should ideally have only one ‘When’ clause that clearly points to the purpose of the test.
• Use past tense for ‘Given’ clauses, present tense for ‘When’ and future tense for ‘Then’.
• Use ‘And’ clause to add multiple Given and Then steps.
• Put scenarios prerequisite steps in ‘Background’. Note: If the prerequisite steps are more technical, then use Before hook.
• Scenarios should be more functionality oriented rather than UI/UX actions.
• You can also write BDD Styled Acceptance Criteria for web services. However, you need to ensure it is understood by the product owner.
• Write scenarios after talking to business people.
• Use declarative steps rather than imperative.

Declarative

Given I pass the header information for SSN
When the client request the URL with data
Then the response code should be 200
  

Imperative

Given I pass the header information for SSN
When client request POST "<ServiceURL>" with json data "<RequestFile>"
Then the response code should be 200
And the SSNcached result should be same as valid transaction response "<ResponseFile>"
  

Tagging

Grouping test scenarios from different features is must. In order to select the tests for different execution types, tagging is important. Nowadays, we have many test cases managements plugins for Jira. Updating automated test execution result in Jira is vital. Managing manual and automated executions result in test case management tool is helpful to collect test metrics and improve the test coverage. As one of the leading test automation companies, we tag BDD scenarios with Test ID, environment (qa, stage, 7 prod), and testing purpose (Smoke, Integration & regression). Using Test ID, we can push the automated test execution status in test case management tool.

Feature Narrative Description

Feature: Title (one line describing the story)
 
 Narrative Description: As a [role], I want 
, so that I [benefit]
  

Step Definitions dos and don’ts

• Don’t call a step definition method from another method.
• Never hard code any test data in step definition.
• Learn how regular expression is used to match a Gherkin step and a Step Definition method.
• Chain the assertions to enable code readability.

assertThat(fellowshipOfTheRing).filteredOn( character -> character.getName().contains("o") )
                               .containsOnly(aragorn, frodo, legolas, boromir);
  

Use Cucumber Lambda expressions to simplify the step definitions.

Given("I login as (.*)$",(String name)-> System.out.println(name));
  

Reporting

BDD Reporting is important to trouble-shoot the failures and collect test automation metrics. You can use Serenity BDD reporting, ExtentReporter for BDD, Report Portal, and Allure Reporting.

In Conclusion

We, as a test automation services company, use Cucumber, Behave, SpecFlow, and Serenity BDD frameworks for our automation testing projects. Following automation testing best practices has helped us to deliver high-quality test automation scripts to our clients with precision. We suggest that you prepare a training plan for your automation testers and make them understand how anti-patterns impact test automation. Writing BDD scenarios which are too long will not add value to the team and it increases script maintenance. If your team is comfortable using DSL, then the output of your test automation scripting will be concise, which will enable the effective collaboration between Developers, Testers, and Business.

All about Karate Framework

by admin | Feb 7, 2020 | Automation Testing, Fixed, Blog | 1 comment

People think of automation to gradually reduce time-consuming repetitive and manual work. It is a known fact that automation testing reduces time, improves productivity, generates reports automatically, etc. Fundamentally one should have some level of expertise in coding to write automation test scripts and subsequently run it without any hassles, you need to have so many things handy while setting up a framework, say for instance you should have technical know-how of integrating necessary plugins, creating a proper folder structure for your code, selecting tools and IDE’s, etc. A newbie software tester with less coding knowledge will be overwhelmed with these setups and potentially neglect automation testing.

We’ve seen a lot of testers, QA engineers using Postman/Insomnia/Paw to test their REST API endpoints manually because they do not know how to automate them. Automating API testing is a more complex procedure, setting the initial process and running it successfully is often one of the most challenging parts of the process, one must be ready to face a series of challenges like parameter validation, a sequence of API calls, testing the parameter combination, etc.

Our engineers gave it a go with many API frameworks like RestAssured, SOAP UI, Katalon Studio and Karate. We have always been excited while working with Karate framework because Karate framework is promisingly a mature framework since API tests are written using plain text Gherkin style as it is built on top of Cucumber and is written in Java.

Why we recommend it

Java knowledge is not required and even non-programmers can write tests with ease ( it runs on JVM)

It supports JSON and XML including JsonPath and XPath expressions,

Re-use of payload-data and user-defined functions across tests is possible.

Simple latency assertions can be used to validate non-functional requirements like performance requirements. (for ex. it is used to measure the performance of the API when it is executed multiple times by measuring the response time)

How to Setup Karate framework

Let us see the step by step guide on setting up Karate framework

Start up your favorite IDE. (we are taking InteliJ IDEA in this example.)

Go to File> New> Maven Project and take the defaults on the first screen.

Under New Maven Project, click on Add Archetype

Enter the following info:

Archetype Group Id= com.intuit.karate
Archetype ArtifactId= karate-archetype
Archetype Version=0.9.4

Click Ok

It should find Karate-archetype. Click on Next.

Enter the ‘Groupid’ and ‘Artifactid’ for the project and click on Next.

Select the maven version and click on Next

Finish the Setup

Now the default karate project will be created with the ‘Feature and config files’.

Karate-config.js

One has to check whether karate-config.js exist on the ‘classpath’. A ‘classpath’ is a place where the important configuration files are expected to be in place by default, karate-config.js contains JavaScript function which will return a JSON object. The values and keys in this JSON object are available as script variables. The below sample JavaScript will tell you how one can get the value of the current ‘environment’ or ‘profile’, and then set up ‘global’ variables and helps you to understand the Karate configuration.

function fn() {   
  var env = karate.env; // get java system property 'karate.env'
  karate.log('karate.env system property was:', env);
  if (!env) {
    env = 'dev'; // a custom 'intelligent' default
  }
  var config = { // base config JSON
    appId: 'my.app.id',
    appSecret: 'my.secret',
    someUrlBase: 'https://some-host.com/v1/auth/',
    anotherUrlBase: 'https://another-host.com/v1/'
  };
  if (env == 'stage') {
    // over-ride only those that need to be
    config.someUrlBase = 'https://stage-host/v1/auth';
  } else if (env == 'e2e') {
    config.someUrlBase = 'https://e2e-host/v1/auth';
  }
  // don't waste time waiting for a connection or if servers don't respond within 5 seconds
  karate.configure('connectTimeout', 5000);
  karate.configure('readTimeout', 5000);
  return config;
}  

A Global variable can be initialized in the config file which can be used in the test feature file. Also based on the environment we can set the global URL from the config file.

package animals.cats;
import com.intuit.karate.junit4.Karate;
import org.junit.BeforeClass;
import org.junit.runner.RunWith;

@RunWith(Karate.class)
public class CatsRunner {    
    @BeforeClass
    public static void before() {
        System.setProperty("karate.env", "e2e");
    }
}
  

Note: By default, the value of karate.env when you access it within karate-config.js – would be null.

Testing the status code

Let’s write a scenario that tests a GET endpoint and checks if it returns a 200 (OK) HTTP status code:

 
Background:
    * url 'http://localhost:3000/api'
Scenario: get all users and then get the first user by id
    Given path 'dishes'
    When method get
    Then status 200   

Here we have the common URL in the Background, in the given path ‘dishes’ will be the route to the URL.
This works obviously with all possible HTTP status codes.

Testing the Response

Let’s write another scenario that tests that the REST endpoint returns a specific response:

Scenario: Testing the exact response of a GET endpoint
Given url 'http://localhost:8097/user/get'
When method GET
Then status 200
And match $ == {id:"1234",name:"John Smith"}  

The match operation is used for the validation where ‘$’ represents the response. So the above scenario will check that the response exactly matches ‘{id:”1234?,name:”John Smith”}’.

We can also check specifically for the value of the id field:

And match $.id == "1234"  

The match operation can also be used to check if the response contains certain fields. This is helpful when only certain fields need to be checked with values or when not all response fields are known:

Scenario: Testing that GET response contains specific field
Given url 'http://localhost:8097/user/get'
When method GET
Then status 200
And match $ contains {id:"1234"}  

Validating Response Values with Markers:

At times, when we don’t know the exact value that is returned, we can still validate the value using markers — placeholders for matching fields in the response.

Ex.
Scenario: Test GET request exact response
Given url 'http://localhost:8097/user/get'
When method GET
Then status 200
And match $ == {id:"#notnull",name:"John Smith"}
And match $ contains {id:”#notpresent”}  

Testing a POST Endpoint with a Request Body

Let’s look at a final scenario that tests a POST endpoint and takes a request body:

Scenario: Testing a POST endpoint with request body
Given url 'http://localhost:8097/user/create'
And request { id: '1234' , name: 'John Smith'}
When method POST
Then status 200
And match $ contains {id:"#notnull"}  

Schema Validation:

Karate provides a simpler, user-friendly and more powerful response validation using Json as a variable to validate the structure of a given payload.

* def foo = ['bar', 'baz']
# should be an array
* match foo == '#[]'

# should be an array of size 2
* match foo == '#[2]'

# should be an array of strings with size 2
* match foo == '#[2] #string'

# each array element should have a 'length' property with value 3
* match foo == '#[]? _.length == 3'


# should be an array of strings each of length 3
* match foo == '#[] #string? _.length == 3'

# should be null or an array of strings
* match foo == '##[] #string'  

We have been using the Karate framework for quite some time now and this framework comes with a very rich set of useful features that enables you to perform Automated API Testing very easily and quickly. It eases the testing process by allowing script developers or testers without any programming knowledge to script the sequences of HTTP calls. We believe that through this tutorial you have gained a lot of useful insights and got a basic understanding of Karate‘s features. Please feel free to reach out for your API testing needs, we are glad to help.

Python Automation Testing for Beginners

by admin | Feb 8, 2020 | Automation Testing, Fixed, Blog | 0 comments

Python Automation testing is gaining popularity these days. As a test automation services company, we have setup a strong training plan to train our automation testers to learn automation testing using Python. In this blog article, you will learn how to start Python automation testing, what are all the tools required, and how to create a test automation career road-map.

Automation Scripting IDE

We use PyCharm Community Edition to create and manage automated test scripts. Download the latest version of PyCharm Community Edition and install. The installation is simple enough. The paid version of PyCharm has more features. However, the community edition is sufficient to build automation framework. There are a lot of good script editors in the market. However, nothing can compete with PyCharm.

Python Installation

When you are installing Python, make sure you select the stable version. Installing in Windows requires a lots of trail and error. First install the interpreter, then pip package, at last install virtualenv package. Use separate Python environment for automation testing. So that you can try new version of Python packages in other environment instead of corrupting the stable environment which is used for automation script execution. Read error messages if any when you are installing Python. You need to know how to setup Python Test Automation code base in other machine or Continuous Integration tool quickly.

Core Python

If you are being trained by someone, please be patient. Don’t rush them to cover Selenium and automation topics. Without Core Python knowledge, it is impossible to deliver high quality automation test scripts. As an automation testing company, we always ensure our automation testers’ scripting quality with a strict review process. Once you are familiar with Core Python, learn Data Structure and Python Algorithms. Never skip this step. If you want to excel in Python Test Automation career, learn how to apply algorithms in programming.

BDD Framework

Behave, Freshen, Radish and Lettuce are BDD frameworks for Python. However, the most popular one is Behave BDD framework. BDD framework enables effective collaboration in your team. Write BDD Scenarios with declarative steps rather than imperative. Tag the scenarios in order to select the tests for different execution types. Before using BDD Frameworks, learn BDD best practices for automation testing. Don’t write BDD scenarios without talking to business people.

Feature Template

Feature: [One line describing the story]
[Optional — Feature description]

Scenario: [One line describing the scenario]
Given [context]
And [some more context]…
When [event]
Then [outcome]
And [another outcome]…
  

Selenium WebDriver

Selenium WebDriver has client library for Python. Learn the below Selenium concepts in order to become an automation tester:

• Launching browser locally and remotely.
• Desired Capabilities for various browsers.
• Windows and frame handling.
• Locators – XPath, CSS Selector, ID, Name, Class Name, and Link Text.
• Selenium Wait Commands.
• Mouse Hover, Drag & Drop, Double-click actions.
• Selenium Exceptions.
• Executing scripts on Cloud Platforms like (SauceLabs, BrowserStack, and Kobiton)
• JavaScript Alert Handling.
• Selenium Event Listeners.
• Executing Selenium scripts from Maven.
• JavaScript Executor Commands.
• Selenium Grid.

Object Locating Techniques

As an automation testing services company, we have mitigated several failed automation testing projects. Most of them were failed due to object locators. If you don’t know how to construct XPath and CSS Selector to identify a webelement uniquely, then you can’t deliver quality automation test scripts. To avoid false positives, object locators should be robust enough. To expertise in object locating, create an HTML file with all available tags, apply styles to them, and embed JavaScript events. Once you are familiar with HTML and basics of CSS, you will get to know how XPath and CSS Selector work in Selenium WebDriver.

Page Object Pattern

Let’s say a step is repeated in multiple methods in a Step Definition file, then it will definitely increase script maintenance effort. Because a small change in the step needs update at multiple lines. When you use Page Object Pattern, you can define all the required objects and action methods in a class file. To create an automated test script, call the Page object methods. Consider an element’s ID is changed, now you need to make the change in Page Object Class not multiple steps in Step Definition.

Create a Base Page Object Class and write all the reusable wrapper methods like click, setValue, and selectDropdown. And then inherit the base class in all the page object classes. The advantage what we get from this approach is if Selenium or Appium release new functionalities which impact your scripts, then you will have to change only the base class not the other page object classes.

Reporting

Behave creates Json report file for you. As a test automation engineer, you can understand what are passed and failed, However, you need to make the report visible and readable to your team. Use Report Portal, Allure Python Report, and Extent Report to create beautiful, interactive and detailed reports for your tests.

Continuous Integration

Once you are familiar with executing Python Test Automation scripts in command prompt, configuring and executing in Continuous Integration tool like Jenkins is a cake walk. Learn how to schedule Python automation testing execution and email the reports from a CI tool.

Career Road-map

After learning the basics of Python automation testing, start focusing on framework development and how to deliver high-quality scripts consistently. Continuous learning is must to stay long lasting in this career. Spend at least 30-60 mins everyday to learn new trends in software testing and automation testing. AI applications are getting released in the market. Learn how to write test cases for AI applications and think how to automate AI test cases.

In Conclusion

We, as an automation testing company, train the new recruits rigorously in Python Automation Testing to deliver high-quality automation scripts. If you are a recent graduate and passionate about test automation field, contact us for more details.

How to start Automation Testing from Scratch

by admin | Feb 11, 2020 | Automation Testing, Fixed, Blog | 0 comments

As we pass by the different industry revolutions the usage of machines in doing the manual activities has helped in their transformation. Similarly, in the software revolution, usage of tools to do the manual testing activity has literally increased the industry throughput due to the following factors.

• Automation doesn’t only help a team in reducing the man-hours spent in business but also helping us the cost.
• With the implementation of automation in a project, we can conduct thorough testing quite frequently to ensure nothing breaks in the system right after quick sprints in the agile model of SDLC.
• It’s quite evident that the system never tires with the multiple executions that we can ensure greater accuracy.

What is Automation testing?

In layman terms it is the usage of tool which can mimic the user action on the software application and perform functional validation without any human intervention or with minimal human intervention.

Levels of testing

Below are a few of test types that are performed as they are required in the Software Testing Life Cycle.

Unit testing:-This is the basic level of testing as soon as a software component is built. The purpose of Unit testing is to ensure that the component does what it is expected to function. This is basically taken care of by the development team and most of it is automated (Example – J-unit, N-unit, jasmine).

System testing: These are the end to end tests that the independent QA writes to ensure the application is best tested and also to uncover any minimal flaw that exists in the system. These tests can fall under UI validation, API validation and DB validation (Example – Selenium, UFT, Test Complete, Tosca, etc.)

Acceptance testing: This is the final level of testing which is taken care of by the Business team to certify the business requirement is met. Mostly Automated scripts developed in System testing will be utilized for Acceptance Testing as well. Acceptance testing scenarios will most probably the subset of System testing scenarios

All these levels of testing can be automated depending on the organization culture and commitment

Types of Automation testing:

Given a careful study of the above different levels of testing it’s time to look at the type of testing in which automation can be achieved.

UI Automation:-This test adds a lot of value to the business considering the fact that the UI is the main layer through which the user interacts more. This is all about impersonating the user actions by clicking on a button, checkbox and sending some information to the text fields and finally saving and navigating across pages. This has widely opted in the business as it has the ability to find the defects that can be foreseen to happen at the user site.

API Automation:- Application Programming Interface is the middle layer between UI and DB. Data is transferred between the database and User Interface is mostly through API calls.

Eg: Rest/ SOAP API tests

Database Automation:- These tests are used to validate the recordset of the database based on certain conditions and with the certain expected results based on certain events.

Eg: JDBC- connecting to the DB and performing validations

Windows Automation:- Given the limitation of selenium- web driver can only talk to browser through http protocol, there are cases where we need to perform some interactions with windows, in that case, we have some test tools which helps us achieve the tasks. We can even integrate with other tests as needed

Eg: Eg: Coded UI, Sikuli, AutoIT

Identifying Automation Candidate:

So far we understood that Automation can help us in lot better ways to conduct different levels of testing in quick time and very accurately. But in order for this approach to be more beneficial, we must take care of certain things before we get into automation.

ROI (Return on investment): – We should understand what way we get benefitted in return when we spend resources to perform the Automation. We gather information like frequency of releases, # of a test to execute, cost to perform one round of test execution and cost to perform a round of automated execution.

Tool identification through Feasibility study: – This is must and can’t be avoided. A feasibility study is assessing Application under test with the set of identified automation tools to validate the best tool to be used for Automating the Application

Application stability: – Ideally in order to automate the application in best standards we must consider its. If there are frequent crashes, Application slowness due to infrastructure issues, etc would not yield the better results that we are looking for.

Setting up the Tool Infrastructure

Below are the points to consider which helps in deciding the tool Infrastructure.

• Nature of Organization, it’s business and their security consideration
• Application types, release frequency, Technology stacks, 3rd party plugin, and customization
• Overall timeline taken for testing / release / Application
• Volume of Test cases / Application and its frequency of usage for each release
• Current tool stack it has from the Test Management perspective
• Environment Management and tool procurement process
• Automation feasibility outcome based on evaluation
• Organization willingness to spend on tools procurement
• Budget required to train and upskill the resources/resource need for Automation

Based on the above pointers and based on the ranking of tool evaluation and other cost consideration Organization has to decide which tool stack it can rely on to maximize the Automation coverage.

Initiate the discussion with the vendor to set up tools in their premise.

Automation Framework:

Automation framework is the set of rules, guidelines, and templates which when implemented helps an automation tester to write the script with ease to get the desired outcome of code re-usability, portability by reducing script maintenance cost.

Pre-requisite for Framework

Below are the components of Framework

• Programming language
• IDE
• Automation tool
• Build tool
• TestRunner
• Dependency management tool
• Version control system
• Reports and logging generation
• CI/CD

Programming language:-This can be chosen by considering the development technical standards and also keeping in mind that whether or not the selection is cost-oriented or an open-source one and finally there has to be a community to overcome certain challenges if we face and support from vendor if it is commercial

Eg; java,c#, python, ruby, perl..etc

Java- widely used because of its open-source feature and rich library and best community than many other languages

IDE:-This is an acronym for an integrated development environment, based on the language and need we should be selecting the proper one to make the assignments hassle free

Eg: Eclipse, Intellij, Visual studio

Eclipse is widely used as it has got more community and rich features and an open-source one, IntelliJ is used mostly as well, it has got open as well licensed editor

Automation Tool:- based on the type of test and type of application and after analyzing the capabilities of a tool we should be picking this.

Eg: Selenium-webdriver, katalon studio, serenity, protractor, Rest API, RPA..etc

Build tool:- Build tools help to pull the latest code from source control, compile it and package it to trigger the execution

Eg: Maven, Gradle, Ant

Maven is widely used given its capabilities and the community it has

Test runner:- When we write some program it’s important to be executed as we need to verify the outcome of what it is being designed for. When developing a framework to trigger tests we should have the best test runner to help with more options to execute tests better such as executing tests multiple times, group execution, passing parameters and so on.

Eg: TestNG, Junit, N-Unit, jasmine are some test runners that are used, but with selenium, it is advised to go with testNG

Dependency management tool- When a sophisticated framework is developed, we may require so many third-party libraries and it is essential to integrate them as well. In order to achieve this, we should have the jar files from those libraries loaded into the binary path of the project, so that the project can recognize them. Loading on to the system may cause a greater problem as it may incur problem when it’s opened in a separate system, how do we overcome?

The concept of dependency management, can help us overcome the problem with the use of the tool Maven or Gradle. It provides a file called (POM.xml) when a maven project is created and we must mention the information of a particular dependency, then it goes and check in the local repository if it doesn’t present it automatically downloads for us locally through the internet. So when we share the project to someone else and opened the POM.xml does the same thing i.e. download things to the system from maven central repository if they don’t exist That way we can ensure the centralization of a dependency across the team. Also, it helps us to have the same and right information in the project.

Version Controlling system:- This system helps us to ensure the changes done to the code base by various teammates are properly merged. With the concept of pulls and pushes to the branches to the repository, we can achieve this. This brings a centralized system where all the persons in the team should look up. There are a few tools which help us doing this

Eg: Git Hub, Bit Bucket, SVN

Reports and logging mechanism:- Without reporting and logging mechanism no framework is completed. While reporting helps us understanding the test results in more easy fashion logging will help us in understanding what happened with a particular test.

Report tool:- Extent reports, allure reports and testing reports

Logging tool:- Log4j

CI/CD:- Continuous integration and deployment is buzzing in recent times as this is believed as a best standard to follow to ensure the code base is built properly and very frequently execution is happening. We can set the goals and pre-actions, Triggers and post- actions with this tool.

Eg: Jenkins, Teamcity

Roadmap for a manual tester to become automation tester

First of all fingers crossed!!!

First please set up a mind that every automation tester today is a manual tester before in some or other way in his/her previous assignments. It’s all about having an open mind to upskill and believing in themselves. Below steps would help in getting that done by self-exercise

Analyze your skillset first as what is known and then what should be learned from the above-mentioned toolset.

Start working on learning basics of a programming language with some real-time tasks for implementation (focus should only be on programming language –java, C# as per choice)

Interact with Automation tester as what concepts are widely used and focus on learning those concepts

Do more practical exercise on the topics that you have learned

Browse and understand the problems faced by people related to the topics learnt and implement to get those refined

Try to study the pain areas and try to improvise code written by using the concepts learned in the programming language (Oops – for instance)

Learn different frameworks available in the market and implement those

Learn other tool integration and practice

Do a mock project covering all the learned topics and demonstrate to the colleagues and get a suggestion for improvement

Provide training sessions on these tools to the beginners and clarify all their queries

Voila, you are an automation tester!!!!