by admin | Jan 12, 2020 | Automation Testing, Fixed, Blog |
Nowadays, we come across many articles related to Cypress vs. Selenium. If you don’t choose the automation testing tool based on you needs, then you will regret forever. Let’s hit the bulls eye – Cypress is not meant to be used by software testers. It is meant for Software developers to write automated Integration tests and Unit tests against a local development server.
If you are a tester and want to use Cypress for automated smoke testing, you can run your scripts on Chrome browser, but not on Firefox, Safari, and Internet Explorer browsers. Reading the main page of the tool and writing the tool review will not suffice the needs of your audience. Product based companies are referring the tool review blog articles before choosing an automation testing tool. A thorough and unbiased tool review will help them to get quick information about a tool.
Introduction
Cypress is a developer-focused tool. If you are a tester, don’t attempt to automate your regression test suite using Cypress.io. In other words, Cypress is a suitable automation testing tool for local development builds. Whatever testing activities you can perform on development environments, you can automate them using Cypress.
Installation
You can install Cypress using npm command or it can be downloaded directly as a Zip file. Note-If you have installed using npm, then you can update newer versions easily.
Scripting Language
Cypress’ scripting language is JavaScript and it is bundled with Mocha and Chai libraries. You can describe your tests and write assertion steps with them. You can also use Intellij IDE to write test scripts. Note: In order to enable ECMA Script 6 support in Intellij, go to Settings->Languages & Frameworks->JavaScript and Select “JavaScript Language Version” as “ECMAScript 6”.
describe('Codoid Review', function(){
it('Test 1',function(){
expect(true).to.equal(true)
})
})
Cypress Selectors
Cypress uses JQuery to find elements in DOM. Why JQuery? Simple. Cypress is for developers. Most of the modern day web developers are familiar with JQuery. It would be easy for them to write locators using JQuery syntax. By default, Cypress polls for 4 seconds to find an element. You can also change the timeout globally or on a per-step basis.
cy.get('.my-slow-selector', { timeout: 10000 })
Project Structure
Once a Cypress project is created, it will consist fours folders – ‘fixtures’, ‘integration’, ‘plugins’, and ‘support’.
- Test data can be stored in ‘fixtures’ folder.
- All your test files (.js, jsx, .coffe, and .cjsx) can be written inside the ‘integration’ folder.
Visual Testing
You can also write visual regression tests using Cypress Snapshots plugin. JQuery Selector finds element, Cypress performs action on the selected element, and the expected result will be asserted. However, if you want to validate whether the page or a web element is displayed as expected to the end user with all the images and applied CSS Styles, then you need to do visual testing.
Browser Support
Cypress supports Electron, Chromium and Chrome. Firefox, Safari, and Internet Explorer browsers are not supported. As Cypress tool is for Integration and Unit tests, running on one browser should be sufficient.
As a test automation company, we use multiple automation testing tools for various projects. We have a conviction that reviewing new tools and subsequently sharing the outcome by means of software testing blogs will immensely help automation test communities to select appropriate test automation tool based on their requirements and tool selection criteria. Contact us for your automation testing needs.
by admin | Jan 14, 2020 | Automation Testing, Fixed, Blog |
Lemoncheesecake is a very well known Python testing framework. As a test automation services company, we have evaluated this framework for one of your automation testing projects. After the proof of concept stage, we decided to use lemoncheesecake framework to write automation test scripts. Initially, BDD support was not available. However, now you can use behave BDD framework in lemoncheesecake. In this blog article, you will learn how to write BDD tests using lemoncheesecake and behave frameworks.
Step #1: Installation
To setup BDD using behave and lemoncheesecake, you need only two dependencies (behave and lemoncheesecake packages). That’s it.
pip install lemoncheesecake
pip install behave
Step #2: Setup LemonCheeseCake Project
Once the required dependencies are installed, you need to create LemonCheeseCake project. There is a command to do that. Run the below command. It will create LemonCheeseCake (LCC) project for you.
Step #3: Create a Feature
Create ‘features’ folder inside ‘myproject’ folder. Create a feature file inside the features folder.
Feature: LemonCheeseCake Demo
Scenario: Demo Scenario
Given I fill login details
When I click login button
Then I should see Home page
Step #4: Step Definition
from behave import *
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
@given("I fill login details")
def step_impl(context):
lcc.log_info("Pass")
@when("I click login button")
def step_impl(context):
lcc.log_info("Pass")
@then("I should see Home page")
def step_impl(context):
lcc.log_info("Pass")
Step #4: Create environment.py
Create environment.py inside myproject folder to install BDD hooks from behave.
from lemoncheesecake.bdd.behave import install_hooks
install_hooks()
Step #5: Run
behave features/demofeature.feature
Once the execution is complete, you can find LCC HTML report in reports folder. The best thing about LCC is reporting. Your team will love it once they start getting reports from LemonCheeseCake framework.
In Conclusion
Why does one need to integrate behave with LCC? We as a software testing company, write BDD and non-BDD automated scripts using Java, Python, and JavaScript languages. Let’s say for instance if u wish to write automated test scripts for data quality checks and UI validations in a test automation framework. Then LemonCheeseCake and behave combination is the ideal choice. LCC framework is gaining popularity among automation testers. We hope that you have gained some useful insights from this article. Every other day, we publish a technical blog article. Subscribe to Codoid’s Blogs to get our updates on the recent technical developments, feel free to contact us for your automation testing needs.
by admin | Jan 17, 2020 | Automation Testing, Fixed, Blog |
Most companies are affected by some problem or the other in new test automation services projects. Keep such windfalls in mind and build stable automation frameworks and ensure a collaborative team effort and own your automation. For automation to be successful, companies should approach it with realistic goals, the right attitude, and a willingness to make it work. Here are eight reasons your automation project can fail:
Unable to describe individual business goals that need to be solved:Return On Investment (ROI) should be understood if you are running automation tied to a business goal, then business leaders will stay invested. Use an open-source tool and hook it into your build server and spend time making your project feature-rich by writing test cases that are aligned to the same business goal.
Automation should be treated as a timed activity and not a process:Make sure developers who build the automation framework hand it over to a competent technical QA team to carry forward. Automation is a software project and should be flexible and easy to maintain. Continuous work needs to be done to update the framework, like new updates, features, and requirements. Most frameworks are tied into other systems like build, metrics, and cloud services, so keep it synced as it evolves.
A team creates it without sole responsibility for standards:Slower test case development by a smaller team that builds in redundancy and regularly reviews and gives feedback to produce quicker value to the project. This team checks requirements, understand business goals, and builds infrastructure to drive the project forward. With strong leadership, an approved roadmap, and strict quality controls, the team should be confident in their tests and its ability to produce reliable results. Each test should be reviewed to ensure it fits, and the team’s QA engineers should be autonomous.
Understand what to and what not to automate:It is imperative to understand that it is good to automate certain functionalities of a webpage and that some scenarios are better tested manually. Only automate stable aspects of your app or web product that is not prone to change but needs to be continuously retested. Save valuable testing effort by automating such repetitive tasks and ensure testers spend time performing exploratory testing.
The team isn’t technically skilled enough to handle the project:Testers should have a certain level of technical expertise to do automation the right way. Such QA team members can be expensive, and not all companies can afford to hire such automation engineers. So hire a few technical experts, but if it isn’t an option, invest in a tool that can help you automate the processes.
Lack of visibility around automation techniques:To be more visible, create an automation document listing all features automated, modules covered, and framework set up. Your method of automation should be clearly defined for the entire Software Development Lifecycle (SDLC). Automation tasks need to be treated separately to make them easier to manage. Make the results visible to the whole team via dashboards and discuss daily progress in team huddles. Conduct sessions that cover different trends, ethical practices, and tools. Make your automation a collaborative effort by involving the complete team in planning and writing it.
Check the testability of the application:Build an application that is easily testable from all (unit, system, integration, and acceptance) levels. Don’t build complex applications that cannot be tested and ensure the testability of a story, feature, or requirement during backlog sprucing, and sprint planning meetings before the development begins. It will alleviate problems later in the SDLC.
Vague automation goals:A robust automation framework seamlessly integrates with Continuous Integration/Continuous Delivery (CI/CD) pipeline, is easily maintainable because it gives quick feedback and runs consistently. Identify two or three high-level functionalities and collect input from automating and stabilizing them first. Separate your automation suites for smoke tests (run after every code check-in) and regression tests (run daily and covers different functionalities).
In summary, these common reasons that cause automation to fail need to be avoided for a successful automation implementation. Keep your projects focused on increasing ROI and business value and hire a highly skilled team or test automation company to ensure a smooth transition. Automation testing companies, like Codoid, can help you in this respect.
by admin | Dec 10, 2019 | Automation Testing, Fixed, Blog |
Automation Testing success can be measured using its metrics. Analyzing only the last execution results will not help you to determine test automation success completely. In order to achieve better results, you need to save all the historical execution results to enable automation testing metrics. For example: if you want to know how many times a feature is passed and failed, then you will have to ingest each execution’s test result in a database and create a web dash board. So that your team can see what benefits we have reaped so far from automated testing.
As a leading automation testing company, while formulating test automation metrics we consider the following two crucial aspects.
- Automation Testing Benefits perspective.
- Script Quality Perspective. if an automated script keeps on providing false-positives, then it will be an indication for automation testing team to make the script robust enough to stop false positives. Let’s see all the required automation testing metrics one by one.
Pass and Fail count from last execution helps to conclude the testing. However, if you get more failures, then automation testers should troubleshoot the failures. If a failure is a false positive, then check how many times the script failed in the past and mark it for the update. When sharing the historical failures count, your team only focuses is to stop false positives.
Automation testing starts from the senior management. They are the one who approve the budget to kick-off automation testing project to reap benefits from it. As soon as the project is started, sharing how many scripts are converted from manual to automation on weekly basis is a vital information for the management team. Consider, 30% of test cases are not automated due to various reasons. your team can plan how much effort is required for manual regression testing for the non automated test cases.
If you have automated test suite, then it should be utilized as much as possible for various testing activities. No. of executions is a great metric for your team and management. Once the regression testing is taken care by automation, the testers from your team will get more time to conduct exploratory testing and to test new features, stories, & other impacted areas manually. No. of times the automated test suite executed metric is an input to your management team to measure the success of test automation.
This metric is useful to enhance scripts quality. How? If you are capturing failure count for Feature, Scenario, and Step level, then you can easily identify automated scripts which are failing often. When your test automation suite is robust enough, then it will add more value to your testing.
Execution duration metric is an input for your team and management. Let’s see how is it valuable to the management team? If everyone in your team is aware of how quickly your automated smoke and regression test suites complete the execution, then your team can take important decision on how many hours/minutes do we need to show up maintenance page during product release deployment? And how many hours do we need to wait to complete the regression testing on lower environments? Even your team can revisit the scripts to fine tune to reduce the execution time.
We recommend you not to execute your automation testing scripts without collecting metrics. Doing so will be a great value add to your project. You can capture many metrics. However, as a first step, capturing the test automation metrics which are mandatory and useful is important. As a test automation services company, we collect metrics to make the effort visible to clients, and the management so as to improve the scripts’ quality.
by admin | Dec 7, 2019 | Automation Testing, Fixed, Blog |
Before starting an automation test project to automate regression test suite and end-to-scenarios, a pilot project will give one a much needed confidence and helps to design test automation framework with all the required features. In this blog article, we would like to through some light on all the key areas an automation testing team needs to focus during a test automation pilot project.
Ensure that you conduct automation testing proof of concept nevertheless you have an expert team with vast experience in test automation. The proof what you are going to collect is for the software under test not for your team.
Test Automation POC steps:
- Identify critical test scenarios.
- Choose a right tool.
- Setup an automation testing framework with basic features.
- Write POC test scripts.
- Capture the challenges faced during POC.
- Execute the developed automated test scripts on the recommended browsers and devices.
- Demonstrate the POC to the management team and stakeholders.
- As a final step, publish the POC report.
Test Automation POC is a checkpoint. If you have got a positive outcome from the proof of concept, then proceed with the next steps. As an automation testing company, we have seen successes in automation testing after conducting POCs and pilot projects. As a next step, identify a subset of Regression Test Suite for the pilot project.
This is a vital step. You need a test automation architect to design a framework which has all the required features to ease automation script development. When you design a test automation framework, ask the following questions yourself:
- How and where am I going to manage Feature files/automated test scripts?
- Test Data File formats?
- How to implement framework to automate Web app, Mobile App, and Desktop App?
- How to use Automation Patterns?
- What are all the preliminary variables required in framework configuration file?
- How does the framework report the failures with screenshots?
- How to collect test automation metrics?
- How to run a script with multiple test data?
- How to handle API automation testing?
A good framework eases script development and enable effective collaboration.
Start automating the identified test cases for the pilot project. Execute the automated test scripts multiple times and make the reports visible to management and stakeholders. A well-planned pilot project gives a sound foundation for automation. If your scripts are not producing false positives and negatives after several executions, you can confidently proceed with full regression test suite automation.
by admin | Dec 13, 2019 | Automation Testing, Fixed, Blog |
In the software industry, there is a constant dispute that surrounds test automation and who can be involved in the design, creation, and maintenance of it. Under various scenarios, there are different degrees of success for the team, product, and company, depending on who within the team does it. When the team uses Test-Driven Development (TDD), developers write code at the unit level to get quick feedback while the QA testers write code to verify higher-level integration layers.
Developers write these unit tests to verify Javascript validation of inputs in the browser and integration tests to check server-side validation of content. QA testers write web driver tests driving the browser for a full-stack validation to ensure that the end-user is talking to the server. The team should code in a maintainable format, and existing tests or any refactoring happens on work in progress, testing frameworks, naming libraries, and patterns used. A cohesive team would build tests according to the pyramid testing stack, and every layer would have different criteria.
The principles for test automation are readability, robustness, and speed. It is a complex process that continually evolves the testing infrastructure along with the product in the development phase. It includes unit tests of the validation in Javascript, integration tests of the server-side validation and data persistence, and a journey test to ensure the new page connects with the data flows as expected. The developers use these scenarios as part of their TDD work, and the QA performs exploratory testing when functionality is available, providing feedback until they meet acceptance criteria. Developers need to be involved in all of the test stacks so that they can refactor existing tests and add new ones without duplication. The QA team provides valuable input on what to test, and the testing infrastructure is built accordingly.
Advantages of a separate test automation team:
- The team shares knowledge on updates to the framework code and review of new tools and libraries together.
- This reduces duplicate code, and most core modules are reused and shared for other tests.
- The team contributes to and uses a consolidated test automation framework.
- Team members work in isolation without daily scrum meetings and can focus entirely on automation work.
- Each team member is assigned to a scrum team, and they help each other out when automation becomes complex.
- The QA team can focus on updating and refactoring the framework to facilitate and simplify the scripting of test code.
As an automation testing services company, Codoid knows the importance of having a dedicated team that not only develops the product but is also involved in the testing process after that. It is ideal if both teams are separate, but it is also imperative that both teams are well-integrated and highly skilled to ensure that times to market and project deadlines are achieved. That’s why we believe companies should focus on their developmental process and leave testing to a test automation services company that uses advanced tools, techniques, and methodologies.
