Blog

A Thorough Jenkins Pipeline Tutorial for Beginners

by admin | Nov 2, 2021 | Automation Testing, Blog, Latest Post | 0 comments

Jenkins has risen to become an expert in Continuous Integration, Continuous Testing, & Continuous Delivery, and assumes a critical role in conveying great applications or items. It utilizes a component called the Jenkins pipeline for achieving Continuous Delivery (i.e.) the capacity to deliver applications routinely for a long stretch. This Jenkins pipeline guarantees that the product is consistently prepared for creation. As a leading QA Company, we have been using the Jenkins Pipeline and found it to be very effective. So if you are looking for a Jenkins Pipeline Tutorial that covers everything from the basics of the pipeline to learning how to perform parallel execution, you’ve found it.

What is a Jenkins Pipeline?

A pipeline in the Jenkins CI/CD can be defined as a series of events or even tasks that are interconnected to each other in a specific order. To put it in simpler terms, the Jenkins pipeline can also be characterized as a set of modules or plugins that enable the implementation & integration of the Continuous Delivery pipelines within Jenkins. It has an expandable automation system that can be used to build basic or complicated ‘template’ distribution pipelines via the Domain-specific language (DSL) used in the pipeline. Continuous Delivery in the Jenkins pipeline comprises 4 major states,

• Build
• Deploy
• Test
• Release

We will be taking a closer look at these states later on in the Jenkins Pipeline Tutorial, but first, let’s take a peek at the many advantages that the Jenkins pipeline CI/CD has to offer.

Advantages of the Pipeline:

• It can divide the jobs into parts (build /test /deploy) and each part can run in each agent.
• Parallel execution of stages is easy to configure and so it can be instrumental in saving time.
• Each stage can execute the different versions of JDK/MVN versions.
• It can be retriggered even from a failed stage.
• Visualizing the build flow becomes possible.
• The build can hold until the user gives the input.
• Version control and code reviews are made easier.
• We can pause and restart the build as and when we wish.
• It will automatically be created as sub-branches in a multi-branch pipeline script.

Pipeline’s Basic Keywords:

Knowing the basic keywords that will be used in the pipeline is the first step that we will be taking in our Jenkins Pipeline Tutorial. So let’s see what these keywords are and how you can use them in the pipeline. We’ll start with ‘Steps’.

Steps

• Steps have to be written inside the stage directive.
• Steps contains the command or scripts that we’ve used in the build.
• One step’s directive should be there in the stage directive.

Stage

• Stage defines a particular stage (build/test/deploy/..) of our job.
• There has to be a minimum of at least one stage.
• The name of the stage will be displayed on the Jenkins dashboard.

Stages

Stage and Stages are two different keywords that you shouldn’t confuse yourself with.

• It contains a sequence of the stages we saw earlier.
• There has to be at least one stage.

Agent

• It defines where we need to run our pipeline script. (Master/Slave/Container)

Stage color

So using color, we will be able to find out the current status of the stage.

• White – The stage hasn’t yet been executed.
• Green – The stage is a success.
• Blue Lines – The stage is being executed.
• Red Line or Red Lines – The stage has failed.
• Red (If in case few stages were a success and one failed, it will show red even if the few have been successful)

Types of Pipeline

1. Declarative pipeline

2. Scripted pipeline

So as seen above, there are two types of the pipeline, and the declarative pipeline is the recent addition that has the more simplified and opinionated syntax when compared to the scripted pipeline. Now let’s take a look at the syntax for these types.

Declarative Pipeline syntax:

Pipeline {
      agent any 
      stages {
         stage(‘Build’) {
              Steps {
             }
        }
stage(‘Test’){
          steps {
        }
     }
stage(‘Deploy’){
      steps {
            }
       }
   }
} 

Scripted Pipeline Syntax:

node {
  stage(‘Build’) {
     }
stage(Test’) {
    }
stage(‘Deploy’) {
    }
}

Variables in the pipeline:

What is a variable?

A variable is used to store values. There are two types of variables, and they are predefined and user-defined variables.

<variable name> = <variable value>

Predefined Variable:

pipeline{ 
    agent any 
    stages{ 
      stage('pre'){ 
      steps{ 
      echo " predefined variable $BUILD_NUMBER $WORKSPACE " 
              } 
       }
   }
}

Output:

User-defined Variable:

User-defined Variable we can define in root level or stage level

pipeline { 
    agent any 
         environment{ 
              MYHOME="Chennai" 
            } 
            stages{ 
            stage('User'){ 
            steps{ 
             echo " userdefined variable $MYHOME " 
                       } 
               } 
         } 
}

Output:

Parameters in Pipeline:

Parameters are used to pass the following types of data dynamically.

• String
• Text
• Boolean
• Choice
• Password
• File

Implementation of Parameters in Pipeline:

pipeline{
 agent any
 parameters {
string(name: 'DEPLOY_ENV', defaultValue: 'staging', description: '') 
text(name: 'DEPLOY_TEXT', defaultValue: 'RnD\nJenkins\nPipeline\n', description: '')
booleanParam(name: 'Are You Have Permission to Deploy', defaultValue: true, description: 'Toggle this value')
 choice(name: 'CHOICE', choices: ['One', 'Two', 'Three'], description: 'Pick something')
 file(name: 'FILE', description: 'Some file to upload')
 password(name: 'PASSWORD', defaultValue: 'SECRET', description: 'A secret password')
}
 stages{
 stage('string'){
 steps{
 echo " string $DEPLOY_ENV"
 }
 }
stage('text'){
 
 steps{
 echo " text $DEPLOY_TEXT"
 }
 }
stage('booleanParam'){
 
 steps{
 script{
 if(TOGGLE){
 echo " now execute, booleann is true"
}else{
 echo " Dont execute, boolean is true"
}
 }
}
 }
stage('choice'){

 steps{
script{
if(DEPLOY_ENV=='staging'){
echo " choice $CHOICE"
}
 }
}
 }
stage('file'){
 
 steps{
 echo " file $FILE"
         }
 }
stage('password'){
 
 steps{
 echo " password $PASSWORD"
          }
      }
   }
}

Output:

Parallel Execution in Jenkins pipeline:

We will be focusing on the parallel build using the Jenkins declarative pipeline in our Jenkins Pipeline Tutorial now. So you can trigger your build system by Jenkins if there are some steps that could possibly run at the same time since they have no dependencies. By following this method, you will speed up your build process and save time for other sequential steps. As one of the best automation testing service providers, we don’t just stop with using the best tools, we also focus on using them in the best way possible.

Parallel Execution:

pipeline {
    agent any
    
    stages {
        stage ('My Java Project Test Scenarios'){
            parallel {
        stage('Unite Test'){
            steps {
                echo 'Unit test Completed'
                sleep 5
            }
        }
        stage('Integration Test'){
            steps {
                echo 'Integration Completed'
                sleep 5
            }
        }
        stage('Security Test'){
            steps {
                echo 'Security Completed'
                sleep 5
            }
        }
        stage('Selenium UI Test'){
            steps {
                echo 'Selenium UI Test Completed'
                sleep 5
            }
        }
    }
              
}
 }
}

Output:

Conclusion

We hope you enjoyed reading our Jenkins Pipeline Tutorial and found it to be informative at the same time. As promised we have covered everything including the basics of the Jenkins pipeline, seen its advantages, explored all the perquisites like variables and parameters that you will need to know to use it effectively. So make sure you don’t just try the Jenkins pipeline, make sure you use it effectively.

The 6 Step Solution for achieving Continuous Testing with Automated Testing

by admin | Oct 26, 2021 | Automation Testing, Blog, Latest Post | 1 comment

Releasing software multiple times into production in a day to deploy the new features/fixes with high quality will need an automation testing setup in all the stages of your delivery pipeline. In DevOps, you have to automate as much as possible to enable continuous testing. So when a product is getting deployed into production more frequently, you have to test the product continuously. For you to proactively search for quality issues, continuous testing has to be enabled across all stages of your delivery pipeline. Since software Test Automation is the key enabler of Continuous Testing, we will be taking a look at the 6 steps that can help you achieve Continuous Testing.

Treating Test Code as Production Code

Whenever a change happens in the product, you have to update your test code to accommodate the change. Let’s say you have to make a change on a common page/screen, your test automation framework should enable you to update the change in that one place instead of updating all the scripts.

Implementing the best practices and design patterns in test automation ease script maintenance. So you have to design your framework in such a way that any script change can be added to the test suite quickly without any hassle.

Kick-off Test Execution for Every Code Commit

Every code commit to version control should kick off automated unit & acceptance tests. When you deliver fast, you will also be in need of quick feedback for the changes or fixes that you have made. Lack of code commit validations will lead to eventual quality issues and regression defects.

Test Automation Framework

You can’t just build the entire framework and immediately start the script creation for Continuous Testing. We have to use the Acceptance Test-Driven Automation (ATDA) approach to develop an automation testing framework. ATDA enables you to write automated test scripts from the very first day instead of waiting for the framework development phase to be completed.

In the past, automation testers used to spend at least two weeks, or even a month in certain cases to develop the test automation framework. The script development itself would begin only after that. So in order to achieve Continuous Testing, you have to start the script development from day one, and then go forward with the product development.

How can we develop scripts without a framework?

Let’s say your team is comfortable with Java. You can use JVM-Cucumber, Selenium, Maven, Appium, and IntelliJ Community edition to start writing automated test scripts from day one. If a script needs a new framework feature, you first have to develop the feature and complete the script. That is how you can evolve the framework and not hold up the script development during the framework development.

Avoid & remove Flaky Tests from CI

If an issue messes with the DevOps pipeline, the entire team should focus on the issue and fix it immediately. Similarly, if a script is flaky, it will make the pipeline unstable. Automation Testing Services is our core service, and we know for a fact that you’ll need a highly skilled team that follows the best practices and uses proper object locators to avoid flaky tests and achieve Continuous Testing.

So if some of your tests are flaky, you must quarantine, schedule, and run them separately, and bring them into the delivery pipeline only when it is fixed and stable.

Test Data Generation

Test data plays an important role in UI, Functional, Non-functional, and Integration Testing. So you have to park adequate test data for test execution. Make sure to avoid failures that can be caused by invalid or missing test data. You should also have an automated system that allocates the required test data and cleans up the consumed data during execution.

Setup the Tests for all the phases

Continuous Testing should be set up in all phases of your delivery pipeline starting from development to production. For example, if a feature is being actively used by the end-users on a daily basis for a particular period of time. You can write an automated test script to check the feature’s usage from the production monitoring data to proactively check for any quality issues. So make sure you don’t just focus on smoke and regression testing as you have to set up tests for different stages in the pipeline.

Conclusion

Being one of the best automation testing service providers, we, at Codoid, follow strict scripting practices and design patterns to avoid flaky tests and have helped many clients to enable Continuous Testing in their DevOps pipeline. Fast feedback is critical for DevOps. You also shouldn’t delay test code development. You have to add the test code as and when new features are deployed into the pipeline.

The 3 Primary Objectives of Automation Testing to Focus On

by admin | Oct 19, 2021 | Automation Testing, Blog, Latest Post | 0 comments

Automation testing is not your departmental goal. Period. So when software testers or automation testers create automated test scripts, they should focus on how the tests will add value to the business or product instead of focusing on increasing the script count. Since analyzing the goals and objectives of automation testing before starting any project is very important, let’s learn about it in this blog.

A Confidence Booster

Automation test executions boost the confidence of your team to release a product. Testing a system manually on multiple browsers & devices is a tiring effort for a tester. Moreover, it is very much prone to errors. Whereas, if rightly done, automation scripts can execute your tests without any deviation. So this ultimately frees up the testers and lets them focus on the problem-solving activities.

In DevOps, there is always a need to automate as much as possible so that the testers can concentrate on Exploratory Testing to unearth bugs, identify new features, and capture edge cases for automation testing. So if there are no application issues, robust and reliable automated tests will run without any hassle, and the passing automated tests will boost the confidence of your team.

Avoid False Positives & Negatives

An automation test script should fail only when there is an application issue. But if your scripts are flaky, you will not be able to reap the benefits of automation testing, and your team will eventually lose confidence in the automated test scripts. That is why we have it in our top 3 objectives of the automation testing list. As a leading automated software testing company, we have seen success in our test automation by following the points.

How to avoid false positives & negatives:

• Recruit talented test automation engineers.
• Train your team to follow the best practices and coding patterns.
• Strictly avoid boilerplate codes.
• Enable the peer-review process.
• Quarantine flaky tests and bring them back to the execution pipeline only if & when they are stable & robust.
• Park the adequate test data required for automated test execution to avoid failures due to invalid data.

You need a highly skilled team to develop robust automated test scripts. We have been providing automation testing services to many of our clients over the years. During this period, we have revived many of the projects that had failed to reap the benefits of effective automation testing. We achieved it by trying to reuse the existing scripts and libraries. But at times, we have thrown away the poorly written automation test scripts into bins. So it is important to keep in mind that if the best practices and coding designs patterns are not followed when creating automation test scripts, you will have to suffer from such scripts forever.

Scripts fail due to invalid test data as well. But feeding the test data before starting the execution is a cumbersome task. In DevOps, automated test executions are kicked off as soon as product changes are pushed into master. So you cannot hold the execution for feeding the test data. Moreover, you will not be able to feed the data a day before the execution if your team is part of the continuous delivery pipeline as they will deliver changes into production multiple times a day.

So make sure to use a test data management tool to upload adequate test data for script executions. Your test automation scripts should be able to pick up the required data from the test data management tool.

Maintenance

When there is a change in the application, you have to accommodate the change in your script as well. If an automated test suite is not changed for a longer period, then it is going to perform the same testing again and again. You should have a process to obtain the below details constantly to keep your scripts up-to-date.

• You should know when a change is introduced in existing functionality.
• Note when a new feature is getting adding to the product.
• Note when an existing feature is removed from the system.

Conclusion

Adding more automated tests to improve coverage is a primary goal of an automation tester. However, if you do it without thinking about the system, goals, and objectives of automation testing, you will be missing out on the many benefits. Automation testing is not an isolated activity and it is pivotal for you to know how your scripting efforts are adding value to the system.

Frequently Asked Questions

An Overview of Gecko Driver & Marionette Driver

by admin | Oct 14, 2021 | Automation Testing, Blog, Latest Post | 0 comments

Gecko is a browser engine for Firefox that has been in usage since the year 1998. But Gecko was initially developed and maintained by a company called ‘America Online’. In the year 2003, Mozilla Corporation took control of Gecko development. Since then, the Firefox browser has improved a lot over the years. Firefox Quantum was released back in October of 2016 with multiple performance and rendering-related improvements. Quantum is an improvement project for the Gecko engine. Being a leading software testing company, we maintain automated test suites for multiple web products. You should be familiar with how Selenium WebDriver works for different browsers in order to achieve multi-browser execution. So in this blog article, you will learn a few important details about the Gecko Driver. (A proxy between WebDriver & Marionette Driver)

pip install marionette_driver

./mach run -marionette

from marionette_driver.marionette import Marionette
 
client = Marionette('127.0.0.1', port=2828)
 
client.start_session()

System.setProperty("webdriver.gecko.driver", "drivers/geckodriver");
    	
WebDriver driver = new FirefoxDriver();
    	
driver.get("https://google.com");

System.setProperty("webdriver.firefox.marionette", "false");

Conclusion

We, as an automation testing company, run automation test scripts across multiple browsers. As stated earlier, Selenium WebDriver is an important tool that will help you achieve multi-browser testing. To achieve that, you should know how each implementation of the WebDriver works on different browsers. When you know the enhancement details of a browser and its driver, you will be in a position to plan and accommodate the changes in the test automation framework well ahead of time without any hassle.

The 7 Best Open Source Test Automation Frameworks

by admin | Oct 12, 2021 | Automation Testing, Blog, Latest Post | 0 comments

Nowadays, we have many open-source test automation frameworks to choose from that we have created this list of the top 7 open-source test automation frameworks to use. But back in the day, test automation frameworks had to be developed using commercial test automation tools. Before the emergence of such open-source automation testing tools, many proprietary tools had the below-listed limitations.

• Only a few limited programming/scripting languages were supported.
• The scripts could be executed only on IE browsers.
• Separate licenses had to be purchased for script executions.
• Only limited Reporting Features were available.
• Third-party test management tools integration was not possible.
• There was a lack of documentation.
• The annual tool support fee was huge.

Even now, some of the commercial’s test automation tools have these limitations. In the earlier days, the Excel-driven test automation framework was very popular. The claim was that testers didn’t have to open the tool to write the automated test scripts. Stakeholders thought that filling the excel sheet to create an automated script was simple and a game-changer. So if any change was required in the framework, it would be done by the test automation architect. That is when the open-source automation frameworks slowly started to emerge in the market.

Selenium brought the capability to automate both Firefox and IE. This was in a time period when even many of the popular commercials automation testing tools didn’t have the feature to automate any other browser apart from IE. It is important to remember that Selenium is a browser automation library; it is not a test automation framework. You have to use Selenium to build the framework.

Why do we need a test automation framework?

With the help of a test automation framework, we will be able to

• Organize automated test scripts
• Avoid duplication of scripts
• Reduce the effort required for maintenance
• Troubleshoot failures quickly with the help of test reports
• Integrate with test management tools
• Manage test data that are required for test execution
• Follow common scripting guidelines
• Ease portability & adaptability

Now that we have seen why we need an automation framework, let’s move over to our top 7 list of open-source test automation frameworks which can be used right away instead of having to build an in-house framework. As a leading QA company, we get projects requests from our clients to work with ready to use framework. Given our expertise in the domain, we believe that this blog article will help you to choose a suitable open-source automation testing framework to kick start script development from day one.

Serenity

First up, we have Serenity BDD which is a ready-made automation testing framework. It has many built-in methods for both Selenium and API automation which will ease the script development process. The Serenity test reporting feature will be very helpful in understanding what tests have been executed, and what requirements have been tested.

We know for a fact that Serenity is a popular open-source test automation framework. Let’s take a look at what makes it so popular.

• Serenity introduces one more layer between the Step Definitions and page objects. ‘Steps’ is the new layer that represents actors or personas who interact with the application. This layer makes your test scripts more readable and maintainable.
• You can mention environment-specific configurations in serenity.conf file.
• Built-in methods and test reporting system for REST API automation testing using REST Assured is a great value-add.
• Serenity supports the Screenplay pattern which helps to produce readable test code.
• It introduces a new Ensure class to write readable assertions.
• You can also run Serenity tests on a Zalenium server to achieve parallel execution.

Robot Framework

Robot Framework is an open-source automation testing framework that has many built-in keywords and libraries. The robot framework’s core is developed using Python. It was developed by Nokia Networks and open-sourced in 2008.

The various standout Robot framework features are:

• It is a keyword-driven framework.
• You can write test scenarios in the Gherkin format.
• It produces both HTML and XML reports.
• You can tag tests to categorize them.
• Robot Framework has standard and external libraries. Standard libraries come along with the installation. Whereas, you can install the external libraries which are required for the project.
• It has plugins for all popular IDEs like (Eclipse, Intellij, PyCharm, SubLime, TextMate, Vim, Emacs, Brackets, Atom, etc…).
• Execution can be scheduled in Jenkins.
• You can use Robot Framework for Acceptance Test-Driven Development (ATDD).
• Desktop App test cases can be automated using WhiteLibrary.
• You can also automate JavaSwing GUI.
• Video recording and screenshots can be captured during execution.
• It supports IBM Mainframe automation testing.

Robot Framework is very vast, and so you will be able to automate any application or API. So even if you don’t find any library for your requirements, you won’t be in trouble as Robot Framework lets you write your own library.

Gauge

Gauge is an open-source test automation framework developed by the Thoughtworks team that eases automated acceptance tests. You can write the test scenarios in the .spec or .md file format. The implementation codes can be written in JavaScript, TypeScript, Java, C#, & Python languages. Gauge allows you to focus only on tests instead of spending time on utilities & keywords development.

If you want to share data from one test scenario to another, you can use the Gauge data store factory. There are three types of data stores (ScenarioStore, SpecStore, & SuiteStore).

If a test execution has any failures, you will be able to rerun only the failed scripts using the below command.

gauge run --failed

You can run these specs in parallel, but you must be able to write the thread-safe test code. Gauge is a simple and flexible test automation framework that avoids complex test code layers.

Atata

Atata is an open-source C# based Web App Automation Testing framework. It provides a rich set of libraries to automate web-related test cases and uses Page Object Pattern to define the UI elements & actions to be performed on web pages. Atata is supported in all major C# based unit & BDD testing frameworks (NUnit, Xunit, MSTest, & Specflow). Atata even integrates ExtentReports for test report generation.

Carina

Carina is a Java-based open-source automation testing framework that can be used to automate Web Apps, Mobile Apps, REST Services, Windows apps, and even DB-related test cases. Carina has one very unique and resourceful feature for mobile app automation testing. It allows you to have common automated snippets for both iOS and Android platforms.

WebDriver.IO

WebDriver.io uses the Selenium, Appium, and DevTools protocol to automate Web apps, mobile apps, and Electron-based Desktop Apps. The DevTools capability was introduced in Selenium 4. However, the WebDriver.io framework has already implemented DevTools without relying on the WebDriver protocol.

Some notable features of WebDriver.io are:

• It is suitable for automating all modern web, mobile, and desktop apps.
• It supports all major reporting libraries (Allure, Concise, Dot, Junit, Spec, Sumologic, etc).
• You can embed video recording in test reports.
• Execution can be triggered based on GitHub actions.
• WebDriver.io has built-in commands. You can also add custom methods if you need any.
• You can schedule executions in Jenkins and Bamboo CI tools.
• You can also migrate the existing Protractor automated scripts to WebDriver.io.

Citrus

Citrus is an open-source test automation framework best suited for integration testing. You can send, receive messages, and also specify control messages for validation. Citrus allows you to test complex messaging scenarios with a step-by-step sequence of operations.

Conclusion

You need experts such as us by your side to provide the best automation testing services. Yes, choosing the right framework is a critical choice. But your team should be capable to create a custom test automation framework, explore, and even suggest the ready-to-use framework for a project.