Blog

Feasibility Analysis in QA Automation

by admin | Mar 1, 2020 | Automation Testing, Fixed, Blog | 0 comments

In this blog we are going to discuss the software test automation lifecycle followed by the feasibility analysis that is required. The feasibility study is a must before taking any action with the automation since it is the deciding factor for automation sign-off based on the value that is going to add to the business.

Life cycle of Software Test Automation

Life Cycle in layman definition is the series of steps/changes that happens in a process to get the desired result.

Life cycle denotes the subsequent actions in a timely fashion. Below are the lists of things to be considered for test automation life cycle.

Feasibility study

Tool identification

Framework identification

Proof of concept

Test scripts design

Test scripts execution

Test scripts review

Feasibility analysis & its significance

As discussed at the start, the feasibility study is something without which we can’t judge the outcome of automation efforts also can’t identify the right automation candidate. This study helps the test team in the forecast on the following activities.

Can the application be automated or not?

What type of tool/ test automation framework can be used?

How much automation is possible?

In spite of high efforts, what’s the value add in automating?

Below are some key parameters while performing the feasibility study, let’s look into each one of them in brief

Feasibility study on the application and understand the functional flow

Before we proceed with automating any application, the automation tester must possess the knowledge of the application. Without having this understanding when any test scenario is considered for automation and we may get into several problems and the task either would be left incomplete or likely delayed.

As an automation tester the focus would be to gain sufficient functional knowledge, get the relevant test cases from the functional (manual) team then conduct feasibility analysis to know how many scenarios can be automated. If at all some cases can’t be automated there must be a valid rationale and the agreement with the stakeholders must be taken.

The analysis helps the test team to showcase their deliverables because after thorough examination team will be in a position to understand what will be covered in automation and how long they require to complete them. Failing to conduct this study could lead to irksome state.

Feasibility study in tool selection

We must conduct a careful study before finalizing the tool. The key points to be considered are

Whether or not the tool can detect all the objects of an application?

The tool that we are going to use for automation must be able to locate all the elements of the application as the locators are the key for the script to interact with the application elements

In case of selenium, we rely on Xpath, CSS selectors, name, Id..etc

If the case is to automate the windows application then selection of a tool that can interact with windows must be chosen

Eg: Coded UI – can locate the windows much better.

Whether or not the tool is best compatible with the application technologies?

There should be a sample POC conducted with all the list of proposed tools to automate the applications then decide which one should be of the good match by considering the application development technologies and its compatibility with the tool.

Eg: while selenium is used to automate all the java, .net, PHP oriented applications, it’s essential to opt for protractor when the application front end is mainly designed with Angular JS. Because angular elements can’t be caught with selenium

Whilst choosing the tool, the cost that would be required for investment should be taken care of. If at all there is a tool exists which does the job w/o any degradation of quality we can fairly go with that as that’s going to add a lot of savings, which will eventually prefer to get better ROI.

Community that’s available for that tool, should be given the importance to an extent as that helps in troubleshooting some of the issues we see in our daily routines.

More the community more preferable the tool is

Feasibility study on framework development

Once the tool choice has come to a conclusion, the next step is to perform the study on framework piece. It should be chosen in a way that the tool supports also it should be quite generic. It should give us better reports to analyze the results

A POC has to be conducted, then after observed the results, we must think of coming up with more reusable components. We might take a lot of hours of time than what is really required for manual validation. But in the long run, as we pass by different phases, the suite adds a lot of value since the execution time will come down to a great level.

A proper check on the regression scripts and baselining them is needed as well because that helps to add more coverage.

Feasibility study on test strategical approach

When it comes to the test approach, the business should take a call whether what between the below strategies should be followed.

Test driven approach

This approach is first to write the tests and then try to accomplish the fix for it. This would be a sophisticated approach and it’s not that easy for non-technical stakeholders to decipher properly. This is completely a call taken by the business based on their budget and other parameters that make sense for a project.

Behavior driven approach

This is buzzing in recent times, as the progress is made by documenting the task in plain English then try to map them with the technical actions. This gives more insight into what a particular test case was written for, what validations are performed. This business model delivers a better report than the above model.

Eg: Cucumber, Specflow, Gauge..etc

Feasibility study on infrastructure availability

In order to proceed with hassle-free automation the test team must not have any issues at infra level, the possible issues that might fall under this category are

Not having access to create project/ branch in a chosen configuration management tool

Not having access to the application/ DB servers to work independently

Software license procurement by the business

Network availability and kind of access to Application, which will delay the execution and cause failures at times

Not holding admin rights to perform certain tasks

The above-listed problems will prohibit automation activities from point to point. Test team must raise the access requests to all the needed things in the project and gain access.

Stable environment and a major part of development should be completed

As a test team we should know when to automate a particular application. If the build quality is at an infancy stage, we see the basic smoke tests are failing then there is no point in automating the test cases at that stage.

When we go for automation when the major development is in progress, we end up adding a lot of maintenance to the test suite as the UI locator elements might get changed often or at times there are some changes expected at functional level, then we may need to re-write the scripts as per new design which is tedious.

Assessing how much automation can be achieved

100% automation is fallacy! Given the complexities of whether or not they are technical or no-technical, that may occur during automation development.

As an automation tester doing this exercise helps the business understand the index of cost and effort saving. Ideally, this is achieved by comparing the functional test cases with the automation test cases. This metric defines what percentage of the tests has been automated and accordingly, the savings were understood.

Calculating the ROI metrics

Metrics are key numbers, which will denote the status, progress, outcome in various ways. As a test engineer/ development engineer when we bother about the accomplishment of tasks, but the management always looks at the numbers extracted as metrics. This is one short and explainable way of presenting data to understand where we are? What have we achieved?

In the same way, in automation world we have certain metrics to be calculated to project the savings to the business, following are a couple of quick examples

Automation savings

This is to understand how much time in man-hours and effort saved because of bringing in automation

Savings = Manual test efforts – Automation test efforts

Return on investments

Certainly we are investing during automation in the means of script development, managing talents and the cost of procuring the infrastructure that’s needed.

Cost of automation investments = Cost of talents + Cost of script development and maintenance + Cost of infrastructure.

Conclusion

More the ROI, the more the profits and that is what business needed as well. In order to achieve better ROI, we should focus on below

Try to bring in more automation

Focus of in-sprint automation development

Right choice of tools, which will incur no or as minimal budget as possible

Develop a generic framework with more reusable components, to avoid extra and often maintenance

Frequently Asked Questions

Customized Test Automation PDF Report

by admin | Apr 16, 2020 | Automation Testing, Fixed, Blog | 0 comments

Generating Customized Test Automation PDF report is a great value add to your team. Let’s say for instance if you wish to share a high-level automation test report with your business stakeholders. Then emailing the report as a PDF is a simple and viable solution. However, sending predefined report or technical information is insignificant and uninteresting to the business. As a software testing services company, we use different report templates for different audiences. In this blog article, you will learn how to create Customized Automation Testing report using Behave & Python.

PDF Report Package Installation

pip install pdf_reports
  

For Windows users: If you see OSError: dlopen() failed to load a library: cairo / cairo-2, then you need to install GTK+. Refer the following URL for more details – Install GTK+ with the aid of MSYS2

Generating Behave JSON Outputs

In this blog article, we are using behave framework JSON output to create PDF report. To create JSON report from behave, use the below command.

behave -f json -o reports.json
  

Consuming the JSON Output

Create a python file and paste the below code.

from pdf_reports import pug_to_html, write_report
import json
file = open(r'../reports.json')
json_report = json.load(file)
html = pug_to_html("template.pug", title1="My report", json_report=json_report)
write_report(html, "example.pdf")
  

Code Explanation

Line #1 & #2 – Package Imports

Line #3 – Opening the json output file which is generated by behave framework.

Line #4 – Loading the json file

Line #5 – Calling pug_to_html method to generate the PDF report.

Note – To generate the report, you need template.pug file. Please refer the pug_to_html method call, you can see template.pug file as one of the arguments.

Template.pug file

Pug is a template engine. The PDF_Reports package uses Pug to generate the PDF output. Let’s see how to create the template file in your project.

1) File Creation – Create & open template.pug file in your project

2) Report title – The below line creates an H1 tag and applies Semantic UI CSS.

h1.ui.header Behave Report
  

3) Sidebar – Creating sidebar in the report which enhances the look and feel.

#sidebar: p Test Automation Report
  

4) Iterate features from the json data

each feature in json_report
    if feature.keyword =='Feature'
        div.ui.piled.segment
            h4.ui.header Feature: {{feature.name}}
  

5) Iterate Scenarios

each scenario in feature.elements
  if scenario.type =='scenario'
  h5.ui.header Scenario: {{scenario.name}}
  

6) Iterate Steps

table.ui.olive.table
 thead
  tr
   th Step Type
   th Description
   th Status
   th Duration			
 each step in scenario.steps
  tr
   td {{step.keyword}}
   td {{step.name}}
   if step.result.status == 'passed'
    td.positive.center.aligned Passed
   else
    td.negative.center.aligned Failed
   td {{step.result.duration}}
  

7) Run the python file which is calling pug_to_html method. If the report is generated successfully, you can see the ouput as shown below.

Full Code

h1.ui.header Behave Report

#sidebar: p Test Automation Report

each feature in json_report
    if feature.keyword =='Feature'

        div.ui.piled.segment
            h4.ui.header Feature: {{feature.name}}
            each scenario in feature.elements
                if scenario.type =='scenario'
                    h5.ui.header Scenario: {{scenario.name}}
                    table.ui.olive.table
                        thead
                            tr
                                th Step Type
                                th Description
                                th Status
                                th Duration
                        each step in scenario.steps
                            tr
                                td {{step.keyword}}
                                td {{step.name}}
                                if step.result.status == 'passed'
                                    td.positive.center.aligned Passed
                                else
                                    td.negative.center.aligned Failed
                                td {{step.result.duration}}
  

In ConclusionThe above template is ( quite simple / very straightforward,) it does not have implementation to handle BDD Scenario Outline, Tags, and Screenshots. As an automation testing company, we use comprehensive Pug reporting template to generate PDF reports. Once the template is in place, you can create multiple reports for different audiences. For successful test automation, make your reports easily accessible to everyone in the team.

Tools & Libraries for Python Test Automation

by admin | Mar 26, 2020 | Automation Testing, Fixed, Blog | 0 comments

In this blog article, we have listed tools and libraries we use for Python Test Automation Services. Few popular tools to automate web and mobile apps are Selenium and Appium.
However, there are few widely used Python Packages which will help you create and manage your automated test scripts effectively.
Let’s see them one by one.

PyCharm-Python IDE

PyCharm is an excellent IDE choice for automation script development. PyCharm comes in Professional and Community editions. We use Community edition since we need it only for automation testing purpose.

Behave BDD Framework

There are many BDD frameworks like (Lettuce, Radish, Freshen, & Behave) which goes hand in hand with Python. We use Behave BDD frameworks for automation testing projects. Behave has all the required features of BDD.

openpyxl

openpyxl is an Excel library to read and write xls & xlsx files. If you want to install and explore openpyxl, please refer the following blog article. How to read & write Excel using Python

Page Object Pattern

When you apply Page Object Pattern, you can enable re-usability and readability. For example: Every test case in your test suite may instruct you to visit login page multiple times. However, when you do this in automation testing, you no need to write Login page steps in multiple test case files. Create a Page Object Class and call the methods whenever you want the login steps. You can write much cleaner test code using Page Object Pattern.

ReportPortal

Creating and maintaining an automation reporting dashboard is never an easy task for a testing team. Writing robust automated tests is a cumbersome task for an automation tester. Creating your own test automation reporting dashboard is an additional effort. We have created many test automation reporting dashboard for our clients. However, nowadays, we have started using ReportPortal instead of reinventing the wheel.

In Conclusion

Python Test Automation is gaining much popularity among automation testers. Follow automation testing best practices, have a right team and use right tools & libraries. This will undoubtedly help you succeed in automation testing.

How to Avoid False Positives in Test Automation

by admin | Apr 7, 2020 | Automation Testing, Fixed, Blog | 0 comments

While building any test automation framework and designing scripts at times we might end up having flakiness in scripts if the proper standards and the test data generation are bad. Given the fact that in test automation framework results are vital as they state whether the test passed or failed and ultimately that concludes the stability of the application under test. Holistically result from reports are the ones which help the business stakeholders to understand where we are with respect to the software product launch.

In this blog we are going to talk about the importance of the results and how should we ensure we don’t fake them. False results are capable of creating a big mess in the system and they are quite costly to fix and learn.

Having said about the fake results, let’s understand what we are meaning to say by the word fake.

It can be of something that has a bug in the software system but the test shows it’s passed. By the term of medical science, this can be addressed as false positive, because we have seen a pass result that is not actually true.

On the other hand, there can be a scenario where the functionality in the system could have been in working state, however, the test shows it failed. This behavior being inline with the medical science term called a false negative.

Relation between automation testing and false positive/negative

Glad that we understood a couple of new terms, but it is still a question as to what’s the relationship between them and the automation testing? why only applicable to automation testing?

If we just recall the purpose of automation, it is essential to script the test case and execute it in the future so that the manual effort spent in reduced. So when we are scripting it for future use a high degree of care should be taken and get the things right. The reason for insisting on the promptness and care is we are not going to be visiting the script thoroughly every now and then. If at all due to oversight something wrongly implemented it obviously serves the wrong results and ultimately leads to flakiness it could be either false positive or false negative.

And to answer the question why it doesn’t applicable to functional manual testing, their user applies his sense than some automated process, any wrong input gets corrected during the execution itself hence there is no room for this concept in manual testing.

Importance to look after false positive/negative

As briefed above at a very high level that these false positive and false negative results can cause a great loss in the business as we are purely mistaking the results. It could impact in the following ways.

A false positive can lead to bug leak in the market as it was pursued to be working good at our site, this can greatly impact the reputation and business loss for the company.

A false negative can lead to spending extraneous effort by various people because of the reported problem when it’s not true. The bug has to be initially understood then learn to repeat then spend efforts to triage the same to conclude it. These efforts ultimately affect the deadline of a release.

Reasons for flakiness in scripts and possible solutions

Let’s understand the prime reasons for occurring false positive/ negative

Incorrect test data

Test data is mandatory to input for any test case to execute. We should always generate the test data as per the guidelines set for the fields in the application. Violation of the rules will lead to abrupt script failure this causes the test case to be failing.

The above failure of the false under the category of a false negative, the reason being we were not supposed to see a failure had there been in data issue.

Duplicate test data

In the recent era applications have gained a lot of maturity in order to serve the user better. Certain important fields will not accept the duplicate data, so if we fail to generate the unique data we again hit a false failure this again comes under the category of false negative.

In order to overcome that we should always generate unique data. One possible solution would be to generate random values and append to the actual test data that is being read from the external excel file.

Improper coding standards

While designing the scripts proper coding standards must be followed failing which will lead to the failure of the scripts in the form of a false positive or false negative. Let’s discuss a few key points in coding standards as what we really mean by and what its contribution in leading to failure is.

Wait handling events

When we are especially automating the GUI we expect some delay for a page to load and the elements to generate and attach to DOM. We also have a parameter of the network, considering all these things we should lay out a plan to handle the weight properly. We should also keep in mind that due to the inclusion of waits the order of n of a script should not go too high. We should be very careful enough to select the appropriate wait, to see below there are 3 waits that we can use as per need

Implicit wait

Explicit wait

Fluent wait

Java wait- thread.sleep(1000)

Initiating and closing browser connections

As we know that the test runs in browser hence invoking and quitting the browser is the most common action that we interact with. The opening and closing actions should be managed correctly by the testNG annotation so that we have a new instance created after each test after each execution as desired. The recommendation would be to launch a new instance after every test so that the impact of one test will not be there on the subsequent test.

Also we should ensure that if at all any browser is running in the port even after completion of test we must close those connections before we re-launch the new browser instance for a subsequent test. It is recommended to use driver.quit() to help withclosing all connections as driver.close() only closes the current browser. This way we can avoid any flakiness that browser can contribute.

Not handling the exceptions

Exception handling brings a lot of sense to the code, it helps application from not to terminate abruptly on failure cases. This is being achieved by surrounding the code snippet with try and catch blocks with necessary exceptions hierarchy.

What basically happens is that, if at all we don’t have a mechanism of catching the anticipated exceptions in the corresponding code block when we face that exception, unfortunately, test case interrupts abruptly and no report would capture that failure as we don’t have that execution caught also any finally block to execute certain mandatory information. In this case, we will get a report with the actual result status of the previous test steps (be it pass/ fail) and report generation stops and it flushes. Ultimately the report doesn’t have the proper results printed in it, its creating false positive or negative in this case.

The prominent solution would be to use the try-catch blocks properly with necessary test step information printed so that everyone can understand that the test actually failed.

Not printing the test step description and status in the report

Report is one of the best evidence to check whether a particular test failed or passed. As it’s a bible it’s essential to capture the right information. We should be very diligent while writing the report and capturing the test step information. A small wording mistake can change the meaning of the result and it creates ambiguity.

Conclusion

Since automated tests are executed by a system and given the fact that it has no intelligence thus far rather than performing the assigned task, it’s our responsibility to get the things right on the very first time, so that the verification and analysis would not take much longer to understand while publishing the results. A small mistake can even be costlier based on the priority so nothing can be misjudged or misunderstood.

Hope the information gave some insights on the topic, thanks for reading through.

Happy learning!!!

What is Test Automation Framework?

by admin | Apr 1, 2020 | Automation Testing, Fixed, Blog | 0 comments

Test automation framework to be explained in laymen terms as the set of guidelines such as coding standards, data handling, report presentation and bringing the repository system into place. With the following of the guidelines, we can make the suite more re-usable with as less maintenance as possible.

What is Test Automation Framework?

Framework is all about defining set protocols, rules and regulations to make the task more standardized.

The framework standard makes us be more procedural oriented. This standard practice will help everyone to understand the way to achieve a specific task and we have great control of tracking things and ultimately we end up have hassle-free maintenance. In the software development life cycle we rely on two we follow two major approaches in order to build a successful software product let’s see what are they and role of the framework.

Let’s quickly discuss the two common framework approaches that are followed most commonly in development and testing streams. These are rule-based approaches for the successful implementation of the task.

Test driven development approach (TDD)

This is one of the best agile development methodologies, in this approach we write tests first then the goal is to make sure the tests go through. This way we can get rid of redundant work and build a very accurate system.

Developers will establish a framework that all unit tests are automated. The code is then added to make sure the tests are passed and then the next implementation takes place. this is the repeated process that takes place until the product is completely built.

Behavior driven development approach

This is also a test-first approach but these acceptance tests were written by either business analysts or testers. Unlike the TDD, this driven approach will try to certify the implemented functionality. BDD Automation Testing approach is built in a fashion that it validates the full functionalities then produces a meaningful report to understand the results.

Role of a framework in test automation

1. Easy to maintain the test suite

2. Easy to transfer knowledge to any new joiner to the team

3. Code re-usability

Components in test automation framework

Thus far we have discussed the framework in a more general way, let’s take a moment to correlate that to the test automation. What are those rules and components that are collectively called as a framework in a test suite?

Dependency management

Whilst creating a project it’s essential to rely on dependency management and build tools to ensure all in the team uses the same version of a particular library. We should never consider loading the libraries from the build path. The omission of this rule can lead to great confusion and we might observe things that are working in one machine, may fairly get failed on other’s end. Also if there are version mismatches due to hardcoding the libraries by every individual we can’t root cause the problems.

Maven & Gradle can help us in resolving the dependencies also to build the code

Reporting results

Results are mandatory to be shared after test execution, so integrating a decent reporting API will help us publishing a report that can be deciphered by all the stakeholders. Report creation must be taken utmost care as it shows how many tests are passed and failed. The failed ones can later be analyzed to see if they are true defects are any script or environment issues.

Best reports that are used are extent reports, allure reports and testing Html reports

Test Runner

In a sophisticated test suite, we will have hundreds to thousand numbers of test cases. Without having a proper test runner tool we can manage the test execution. These tools will help us running the tests with different set options, some are a below

Prioritize the tests

Making one test dependent on other

Execution the same test for the number of times we want

Running the failed test cases

Grouping the test cases and run them in parallel of sequential fashion

Best examples of test runner tools are J-unit, N-unit, testNG, jasmine..etc

Version controlling system

Integration of VCS is essential as we as a team commit a lot of changes to the code base on a daily basis. It’s highly difficult to integrate the changes. Proper education on pulling he code as a first activity before we start our day then pushing at the end of the day after a review will help us maintain the code base up to the date.

Best examples of VCS systems are GIT, BitBucket, SVN..etc

Logging mechanism

Logs are anywhere in the world are treated to be important as that helps greatly in triaging and root cause a problem that’s occurred. We need to write log files to a file for each step we perform during test execution. This log acts as evidence to the malfunction we observe.

Log4j is one of the best log tools to be integrated

Third party tool integration

At times in order to perform specific tasks we might not be able to achieve with the existing framework then we should consider adding any third party tool that can help us doing that, this is how we are making our framework more sophisticated and precise

Example for the third party tools is VNC viewer, AutoIT, winnium…etc. we will need these tools while there is a need to perform some windows actions in selenium framework.

Screenshots capturing

We should have a disciplined screenshot capturing mechanism and storing them in a repository to help with understand the issue that occurred during failures. We can take a call as to whether to capture only on failure cases or on any test step depending upon the memory we have.

Types of test automation frameworks

Given the design implementation and other properties we have various types of test automation frameworks and they are as below

Linear framework

Linear framework is implemented in such a way that the test is directly written in a file as a straight forward test. This test can also be designed by using any record and playback tools. This kind of approach, however, is simpler to design it leaves us doing a lot of maintenance.

Advantages:

1. Easy to develop scripts

2. Not much coding experience is required, only tool experience would do the trick

Disadvantages:

Since the test is written directly, it’s very difficult to maintain the script when there are changes to the applications.

Modular framework

Modular framework is designed in such a way that, tests of common modules were first identified and then design those as reusable or generic methods. Since the common methods were designed as wrapper methods we can very well re-use them in any other test cases.

This kind of approach is followed when the application development is being done as individual microservices. Tests are developed for a specific service and only those can be invoked on demand.

Advantages:

1. Since the common flows have been documented as reusable methods, re-usability increases

2. Less maintenance required

Disadvantages:

Still the data is hardcoded in tests

Data driven framework

Data driven framework has been identified as one of the powerful frameworks across all. In this type, we can service the test data that’s needed via excel, JDBC sources, CSV files. The tests become more generic as the data has been separated from the test code. With the help of various data, script flow can be altered and that way we can achieve more test coverage.

In this we have classified the data as two types as

Parameterization

Any data that looks to be static and we don’t see dynamic changes can be considered as a parameter. The best examples for this kind of data are URLs, JDBC URLs, endpoint details and environment details. We tend to read this data from the properties file or from XML file

Dataprovider

This is the real test data needed for the test case. We basically read this data from the external sources such as Excel workbooks, CSV files, JDBC sources. This data has the ability to alter the flow of the test case. This facilitates us to provide more coverage with minimum code base.

Advantages:

1. Since the data is separated from the tests, tests become more versatile

2. Data can easily be created as per the requirement

3. Easy to maintain the scripts

Key driven framework

In keyword driven framework, the codebase/test case is driven by the action-based keyword. In this framework, we have all the actions identified and list as keywords then map to the application functionality. The keywords again are consumed from an external excel workbook or from a separate java class. This framework can be even developed without application by assuming the functional flow but it needs high expertise and it can lead to a re-work too if things don’t go right.

Advantages:

Since the code base is developed as per actions, methods developed for a particular action can be re-used.

Hybrid framework

Hybrid framework is a customized one, all the best things from the different frameworks are pursued in this. When we say customization, there is no standard practice as such, it’s all about the requirement within a particular project. We might see a mix of data-driven, keyword and modular practices to make the framework more versatile and flexible to maintain.

Behavior-driven development framework

Behavior driven development approach is also known as ATDD (acceptance test-driven development). In this model, the tests are driven by a test step written by either BA or tester in plain English that is understood by everybody. A spec file or feature file is created for a specific test case in gherkin language then the code base is developed accordingly.

How is this model helping?

Ever since the agile model of development has emerged, we started giving more priority to parallel testing and early testing alongside development. There were some challenges in understanding the test case developed in the above discussed other frameworks. Tests appear to be more technical so that it’s getting difficult for non-technical stakeholders to understand. In this approach as the test case written in a specific language which is gherkin (more or less English). Given the design, it’s easy for everyone to say what’s happening with the test also report makes more sense too.

Examples of this kind of frameworks are- cucumber, gauge..etc

Conclusion

There is no wrong in being a process nerd because unless we have protocols defined even in our lives we can’t be sure of the progress and get the things right on the very first opportunity. In the same fashion with the help of above-defined rules we can achieve the tasks hence thanks to the framework and its defined best practices. We should still be under an impression that the best yet to come, given that sense we expect lot more enhancements to make our lives far easier at work. Change is inevitable be ready to adopt.

Thanks for walking through this long, I reckon you would have had some learning.

Python Behave BDD Framework Overview

by admin | Mar 28, 2020 | Automation Testing, Fixed, Blog | 0 comments

Python behave is a much widely used BDD framework these days. In this blog article, you will get to learn behave BDD framework’s features and how to use it to create automation test scripts. Behave is an open source tool which has 62 contributors who are actively developing new features and fixing the issues. Python Behave is a matured BDD framework.

Implementing readable and business friendly automation testing solution is the need of the hour. When your automation test suites gives provision for non-technical people to collaborate, it will be a value add to your product/project. As an automation testing services company, we get lot of inquiries on implementing business friendly test automation solution of late.

Feature Files

behave is considered to be a clone of Cucumber BDD framework. Most of the features of behave is similar to Cucumber. However, there is a provision for additional flexibility in behave framework. Let’s have a look at them one by one. You can execute feature files in the following ways – by providing feature name, feature directory, at last without providing feature details. When you run behave command without feature path, it will search feature files in features directory by default.

Fixtures

Fixtures aka Hooks are used to call Setup & Cleanup code before and after Test Run, Feature, Scenario, and Tag. You needn’t write a separate method for ‘After’ hooks. Both ‘Before’ and ‘After’ hooks can be implemented inside a single method. Let’s look at the below example.

Feature

@launch.browser
Feature: Codoid Website Feature

Scenario: Contact Codoid Team
  Given I am on home page
  When I submit contact us form
  Then I should see Thank You page with a message
  

environment.py

from behave import fixture, use_fixture
from selenium import webdriver

@fixture
def launch_browser(context, timeout=30, **kwargs):
    context.driver = webdriver.Chrome(executable_path='driverschromedriver.exe')
    yield context.driver
    context.driver.quit()

def before_tag(context, tag):
    if tag == "launch.browser":
        the_fixture = use_fixture(launch_browser, context)
  

step_def.py

from behave import given, when, then
from selenium import webdriver

@given('I am on home page')
def step_impl(context):
    context.driver.get("https://codoid.com")
    assert True is True
  

Executing Remaining Steps in a Scenario

By default, scenario execution will be stopped if any of the steps is failed. However, you can override this in behave framework using Hooks and Command line.

Hook

from behave.model import Scenario

def before_all(context):
    userdata = context.config.userdata
    continue_after_failed = userdata.getbool("runner.continue_after_failed_step", False)
    Scenario.continue_after_failed_step = continue_after_failed
  

Command Line

behave -D runner.continue_after_failed_step=true features/
  

Filtering Scenarios and Examples using Tag Value

You can select scenarios and examples using Tag and its value. This feature is helpful when you want to select an example based on the environment value for a tag.

Feature:
  Scenario Outline: Wow
    Given an employee "<name>"

    @use.with_stage=develop
    Examples: Araxas
      | name  | birthyear |
      | Alice |  1985     |
      | Bob   |  1975     |

    @use.with_stage=integration
    Examples:
      | name   | birthyear |
      | Charly |  1995     |
  

In Conclusion

As a test automation services company, we advocate behave BDD framework for creating readable and understandable automation test suite. It is still widely used by the automation community and its contributors are releasing newer versions frequently based on the market need.