by admin | May 18, 2020 | Agile Testing, Fixed, Blog |
The main discourse in this blog is to understand the automation testing pyramid and to match that with agile work model. Automation testing pyramid defines the types of tests to be conducted at various levels, their scope and how intense the tests should be. Let’s take a moment to understand about the testing pyramid and how automation testing is performed under the governance of testing pyramid.
Test Automation pyramid and its principles
On a brighter note, Automation testing pyramid is nothing newer than what we have studied in v-model that is mapping a corresponding testing in accordance with the development readiness. But the only difference that we see is the v-model strategy doesn’t tell how intensified a test type should be, but that is very well explained in testing pyramid theory also it emphasizes the value of test automation in each area.
Testing pyramid says that it’s best way to conduct through testing in the early life cycle such as in unit and integration tests in order to prevent the bugs then a few end-to-end tests that cover most customer likely scenarios.
Let’s deep dive in about each level of testing in a hierarchical fashion and see what testing type should kick start and how much focused that test should be and what’s the value that automation is adding.
Unit testing
We know that in agile model the development and testing should go in parallel. In agile the development is managed in small component level initially then they integrate and after they ensure the system is developed to ensure the feature is developed.
The best way to ensure the most of the unit tests are automated since that’s the test which can give us the feedback very early also they are easier to fix, because the bug that is seen is likely happening due to the code error present in only that specific module. These tests can be run more frequently i.e. as and when there are deployments of that module we can start execute the test cases. The degree of dependency exists for this test is relatively small.
Integration testing
In the midst development team integrates these smaller components to perform business logic. To test the communication between the services we must consider writing some powerful integration tests.
Basically these tests are conducted at API layer. This integration testing phase is vital because in the previous stage we have ensured that a component or service works better as a standalone but it could break when it is being integrated, so in order to verify that the communication is proper through API calls & business logic is performed and finally the information has stored in database. We need to write these tests.
System testing (end-to-end)
Now we have done testing at two levels, ensured the designed services were working intended when they were as standalone also they were performing needed business logics after successful integration. If we think this testing is needed we are missing something great here. For a moment when we go back to fundamentals of testing the high level definition says, the goal is to verify that the functionality is working as intended and it should serve the purpose of the requirement.
We need to consider the user journeys that are key and what we think that are executed in the field as often as possible. Automation can add great value here as well, because the execution in the end to end phase most likely happens on UI layer and it can help us find any defect before the application goes live. Being in line with the test automation pyramid there shouldn’t be too many end to end tests because these tests however can help finding defects, they add significant delay to feedback results due to the fact that these tests have to interact with so many elements and pages.
The fact is that though we find a defect it’s very costly to fix. When we say it’s expensive, it is mainly due to two things those are effort needed to identify where that particular bug occurs then understand the bug then fix it, the second one is it is found very late in the cycle. After given a fix for the identified bug a thorough regression testing is needed as we never know that the new code change will impact any other system which would have been working before.
Testing pyramid followed in manual testing
In case of manual testing the approach is followed as per the below figure, where developers write unit tests to ensure the module works, but these tests are very minimal in number. As we go past different level of testing levels we intensify the testing we start writing more tests. Finally we end up conducting through end-end testing by testing all the customer journeys or use cases manually.
The main drawback of this approach is that, we are finding the defects but not preventing. Yes, you read it correct, the meaning for that statement can be explained as since we are not conducting through testing there could be fair chance for a particular micro service or component that is being developed may break on certain edge cases and this was hidden and when it was found later the developer had a lot of code base to look into to understand what causes the bug it apparently takes more effort than it ideally should have taken during initial stage. Moreover we require lot of regression testing to ensure the given fix hasn’t impacted any other working module.
Holistically by following the anti-cone approach we are welcoming lot of redundancy by choosing the nook to corner testing at end-2-end phase. The amount of effort needed to write automated test cases for end-to-end is vague, also the amount of time that they take to execute and provide results is relatively higher. Its lot of overhead to do that at this phase as it has high chances to skew the deadlines towards right.
Conclusion
If we were to brief the and re-iterate the chronicles of the test automation pyramid.
1. Start with an intensified automated testing early in the life cycle probably at unit testing and integration testing levels
2. Don’t just limit the automation to the UI layer, we must consider automating at API layer as much as we can, given the TDD approach of development in agile SDLC model.
3. Don’t overload the test pack with too many end to end test cases unless they think they are likely executed by customer and it has some important business logic to perform, believing that the other requirements would have been covered under user story based testing in earlier phases.
4. The reveres cone approach is only helpful to find the defects but not to prevent them for occurring from earlier life cycles.
5. Devops integration is best matched with the test automation pyramid it helps both the development and testing teams to trigger their tests.
Thanks for walking through the content, hope there would have been some learning.
by admin | Sep 9, 2020 | E-Learning Testing, Fixed, Blog, Top Picks |
Education is essential, and it should be open to everybody. As innovation pushes ahead, individuals are attempting to build the right platform for online training. During offline training sessions, you can visually connect with educators. It’s getting simpler to convey, pose inquiries, and collaborate, and so on. With a pandemic in its full rage, employers are increasingly opting for online channels. It appears to be that eLearning platforms attempt their best to give quality learning experience. Yet, it seems something is deficient in an informative online session. Consequently, there is still space for advancements in eLearning apps.
E-learning testing is essential and could be done quickly by yourself or using E-learning testing services. Regardless of whether you have spent endless hours building up every segment, it’s consistently advantageous to give it a careful tweak before you roll it out for your employees or students. Here are 14 things to double-check before launching your e-Learning course.
1. Incorporation of Learning Objectives and Goals
It’s smart to go over your whole eLearning course and ask yourself whether each component you’ve included, from the content to the illustrations, serves your learning objectives and goals. Everything ought to be meaningful. It must help your students in grasping the topic, instead of diverting them from the eLearning experience.
2. Each learning Unit is Independent
Each learning unit you’ve made (which is an exercise or module that can uphold its learning objectives and goals) should provide your students with insights towards a given theme. Along these lines, ensure that your learning units’ entirety is independent and that they fit into the master plan of your general eLearning course.
3. Syntax, Punctuation, and Spelling
Triple check your eLearning course to guarantee that your commas, periods, and marks are in their legitimate spot. Additionally, check the spelling of each text as even those that are important for a video or a picture, to ensure that everything is right on target. Keep in mind, spelling and language errors may appear to be unimportant, but have the potential to decrease the general nature of your eLearning course by causing it to appear to be less tenable and composed.
4. Brand Joining
Is your eLearning course in-accordance with the brand picture of your customer or your eLearning firm? Is it intelligent of the topic you’re attempting to make, and does each segment help create a powerful and vital learning condition for your audience? Answer these questions for yourself before proceeding further.
5. Course Diagram/Progress Markers
Are your students ready to keep tabs on their development while partaking in the eLearning course? Incorporated markers that they can use to rapidly and advantageously perceive how far they’ve come, what subject they’ve discovered, and what different exercises they will explore soon. It will give them that additional increase in inspiration and will push them to work more.
6. Correct and Clear Navigation
The route of your eLearning course ought to be as transparent as it could reasonably be expected. It should incorporate forward and in reverse route symbols, a home catch, and a connection to take students to the eLearning course diagram screen. Additionally, ensure that each button is dynamic and show up in a similar spot on each page to dodge confusion.
7. Valuable Content
Each bit of content should reflect practical applications and the necessity of the topic. If not, your students won’t see the genuine benefit of finishing the eLearning course and, like this, are not prone to participate in the learning cycle.
8. Constructive Feedback
When students complete a branching situation, recreation, or appraisal, would you say you are furnishing them with viable criticism so that they are benefitting from their experience? In a perfect world, your students need to be assessed consistently so that they can learn from their mistakes. Guarantee that you are utilizing efficient criticism tools and strategies to give your students the help they need.
9. All Programs must uphold Media (videos, images, and audios) Components
Students will be getting to your eLearning seminar on a wide range of devices. It is why it’s essential to confirm that your multimedia components can be seen on an assortment of devices. Are your sound and video advanced for web seeing? Will they, despite everything, be successful learning apparatuses on the off chance that they are seen on various devices?
10. Links are Dynamic and Vital
Double-check that your links’ entirety is dynamic and that they will show up in another popup window. It can be undertaken after you’ve launched the eLearning course to ensure that all the links are still dynamic.
11. Is it Mobile-Friendly?
View your eLearning seminar on various sorts of cell phones, if conceivable, to confirm that you can get to the content entirely. Likewise, guarantee that it, despite everything, your application holds its responsiveness and tasteful intrigue when it’s seen on a little screen.
12. Comprehensive Assessments
Have you made exams, tests, and quizzes that are genuinely going to ascertain student intellect? Have you utilized the correct inquiry design (numerous decision, valid or bogus, etc.) in light of the topic? Do your inquiries offer the perfect measure of a challenge for your students, or would they say they are excessively difficult or easy? Therefore, while gauging your students’ comprehension, ensure that you keep the questions mentioned above in mind.
13. Lesson Synopses
Make sure to incorporate recaps or synopses toward the end of every module. Also include brief outlines toward the start, with the goal that your students know about the essential thoughts or standards they will gain from every exercise.
14. E-learning Course Testing after the Launch
Once your course is delivered, it is essential to verify the working by genuine clients. Have them test the application and give their criticism. It will permit you to distinguish any errors or features that would require improvement.
To Wrap
Utilize this list to remember everything to double-check before launching your eLearning course, to guarantee that your eLearning course is holistic and perfect. Keep in mind that while this cycle may require time and exertion, it will spare you the cerebral pain of rolling out these essential improvements after the course’s launch.
by admin | May 8, 2020 | Software Testing, Fixed, Blog |
CSV stores data in tabular format. Importing CSV files can be achieved at much faster, rate with less memory consumption. In this blog article, we have listed a few Python CSV File Read Examples for your better understanding and reference. We, as a QA company, use large CSV files to verify data quality between source and target. In order to automate data quality checks, we use Python libraries to import CSV files programmatically.
Built-in CSV Library
Python has built-in CSV library. You can read & write a CSV file. You needn’t install additional packages to use this module.
import csv
with open('test_data.csv', 'r') as file:
reader = csv.reader(file)
for row in reader:
print(row)
Read CSV using Pandas
Pandas is a powerful data analysis Python library. Let’s say for instance if you wish to import large CSV files on one go for Pandas. We, as an automation testing company have used Pandas library for our ETL & Data Analytics projects at various occasions and found it noteworthy.
import pandas as pd
test_data = pd.read_csv("test_data.csv")
#Reading First Value from name column
print(test_data["name"][0])
Convert XLSX to CSV
If you want to convert XLSX to CSV in Python programming, you can try out xlsx2csv library. It converts large XLSX file to CSV.
CleverCSV
CleverCSV detects the dialect of a file automatically , additionally it has command line implementation, if you would like to check the file outside Python environment.
# importing this way makes it easy to port existing code to CleverCSV!
import clevercsv as csv
with open("data.csv", "r", newline="") as fp:
# you can use verbose=True to see what CleverCSV does:
dialect = csv.Sniffer().sniff(fid.read(), verbose=False)
fp.seek(0)
reader = csv.reader(fp, dialect)
rows = list(reader)
In Conclusion
When you use Python libraries, you can craft your test automation scripts quickly and concisely instead of writing lengthy snippets. The above listed libraries will be helpful when you handle CSV files. We, as a software testing company, use various Python packages for ETL & Data Analytics testing projects. There are several other Python CSV File Read packages available in the market, However, they are not active in development. We hope you enjoyed this blog article and found it worthwhile.
by admin | May 13, 2020 | Agile Testing, Fixed, Blog, Top Picks |
Agile is test first model, this defines how to kick start testing in parallel with the development. The main agenda of this blog is to relate the automation testing to this model. The discussion is mainly on how to develop the test pack model so that it best suites the agile velocity and with the sprint level expectations. We, as an automation testing company, create a concrete plan while bringing automation into Agile, conduct proper analysis then understands the feasible fashion of test script development.
Designing Automation Testing Framework Given the expectation that the testing has to go hand in hand with automation we must have a proper plan of developing scripts. Whether n-sprint approach or n-1 sprint approach to be followed? Let’s quickly look at how those sprints work.
In this model, we will have to consider all user stories that are listed must be automated.
In this model we likely trail by a sprint, whatever the stories those were developed in previous sprint are automated in current sprint. Despite the model, we should always make sure that the regression pack is kept up to the date in the same fashion we have to maintain the sanity pack too.
Types of Automation testing When in agile, we build the system in many iterations and we call them as sprints. It is obvious that we will have only smaller chunks built in linear fashion. The question here is – how do we accommodate the space for test automation when UI and the whole system is not available? Because when automation testing is said, 90% of the spectators talk about automating at UI layer as that has ROI which is better.
If we think automation testing is best deployed to test UI, not others we are missing greatly the ideology. What we are meaning to say here is that we should think of automating at different layers such as API & DB validations to ensure that the system is doing the right thing. If we can ensure that at the initial stage our job would be lot easier, also it helps the business greatly as we are uncovering the bugs early in lifecycle and which are not that expensive to fix.
For an illustration let’s consider that the development approach is a TDD one, we will not have the complete system to test, rather we will have smaller modules/micro services would be developed in an incremental fashion. To support the quick testing at that level we should consider designing a automation testing framework that can have tests based on module wise. We also need to understand what are the services offered by the component and accordingly write type of tests.
Write powerful integrated API tests in order to verify the micro service – micro service communication. Perform some data base table’s validation to see the business logic has been processed successfully and information stored at right table. We will apparently not have UI developed initially so as soon as that gets ready we should be ready in testing the flows that starts from UI.
Test Automation industry has taken a big leap from documenting the manually tested functionality as scripts for future execution to the purpose of serving the need for in sprint automation due to the agile parallel test execution expectation. With following the discussed practices and understanding the strategy of development and testing need, we can design a matching automation testing framework that serves the need of the hour. There is no standard or hard coded practice as such, rather it fully depends on the context of a specific project.
by admin | Apr 24, 2020 | Selenium Testing, Fixed, Blog |
Bazel is one of the most widely used open source build tool these days, the rationale behind using Bazel is aplenty. Let’s say you want to compile Java and Python projects into one build file, then Bazel would be the ideal choice. As a QA company, we use Maven, Gradle, and Grunt to compile & kick-off automated test suites. Bazel is something new for us. In this blog article, you will learn how to run Selenium Scripts using Bazel.
Advantages of Bazel
1) Platform Independent: You can run your Selenium scripts on Linux, macOS, Windows.
2) Compiling Large Files: Bazel can handle large files easily.
3) Languages: You can build and test C/C++, Objective-C, Java, Python, and Bourne Shell scripts in one build file.
How to run Selenium Scripts using Bazel?
Bazel has its own build language. Using Bazel’s predefined test rule, you can set automated test scripts in the build file irrespective of the programming languages used. We, as a test automation company, explore new technologies with the intend to ease automation testing process. Bazel is so programmer friendly, even Selenium developers are using Bazel to test Selenium’s features before deploying. In this blog article, we have used Windows 10 and Python to show the code execution. Let’s start from installation.
Prerequisite – Visual C++ Redistributable for Visual Studio 2015
Visual C++ Redistributable for Visual Studio 2015 is a prerequisite if you are installing Bazel on Windows 10. Use the following link to install the package – Visual C++ Redistributable for Visual Studio 2015
Bazel Installation on Windows 10
Step #1 – Download bazel--windows-x86_64.exe from the following link – Download Bazel
Step #2 – Rename the downloaded file to bazel.exe
Step #3 – Copy and paste the bazel.exe wherever you want.
Step #4 – Set the bazel.exe path in PATH environment variable in Windows 10.
Step #5 – Restart your PC
Create a Python Project
Step #1 – Create a Python project
Step #2 – Create a directory called ‘scripts’
Step #3 – Create a directory called ‘drivers’
Step #4 – Download and paste the chromedriver.exe in drivers folder
Step #5 – Create a python file and name it as test-1.py inside the scripts folder
Step #6 – Paste the below snippet inside the test-1.py file
from selenium import webdriver
from bazel_tools.tools.python.runfiles import runfiles
r = runfiles.Create()
driver = webdriver.Chrome(executable_path=r.Rlocation("test_suite/drivers/chromedriver.exe"))
driver.get("https://codoid.com")
driver.quit()
Note: In line #4, the chromedriver executable path is set using Bazel’s runfiles module. Because all the supporting & testdata files for automated scripts need to be accessed using runfiles module. Maybe in future enhancements we will have a simple workaround instead of calling runfiles.
Creating WORKSPACE file
Bazel identifies a directory as Bazel Workspace when it contains WORKSPACE file.
Step #1 – Create WORKSPACE file without any extension
Step #2 – Paste the below line to name the workspace
workspace(name = "test_suite")
Creating BUILD file
Bazel build file is being used to compile and run your test code. We, as a software testing company, use Maven POM.xml files to compile and run test code for Java projects. If you have test code base in different programming languages, then you can opt for Bazel to compile & execute test code of different languages into a single build file. Let’s see how to create a build file.
Step #1: Inside the root directory, create a BUILD file without any extension.
Step #2: Copy and paste the below snippet in BUILD file.
py_binary(
name = "test-1",
srcs = ["scripts/test-1.py"],
srcs_version = "PY2AND3",
data = [":drivers/chromedriver.exe"],
deps = ["@bazel_tools//tools/python/runfiles"],
)
Code Explanation
Line #1: py_binary is one of the Bazel’s rules. For Python, Bazel has two more rules – py_library & py_test. Any Python related tasks can be defined using these three rules in the build file.
Line #2: You can name your tasks using ‘name’ attribute.
Line #3: ‘srcs’ attribute is used to mention source files inside the rule. You can also mention multiple source files.
Line #5: To launch the chrome browser, we need chromedriver.exe file. So we are copying the file inside the runfiles folder which can be found in bazel-out folder. Use ‘data’ attribute to copy all the supporting files.
Line #6: To access runfiles in Python code, you need runfiles module from bazel_tools. Refer Line#2 in test-1.py file, you can understand why we are mentioning this dependency here.
Bazel Run
Now it is the time to run the script. You can run the rules from command prompt. Let’s see how to run our test-1 task.
Clean – bazel clean
Run – bazel run :test-1
That’s it. You can see browser launch after running the above commands. We have uploaded the Python project in Github. Please refer it if you face any issues. Full Code Download Link
In Conclusion
As a test automation company, we manage and maintain automation test scripts in multiple programming languages. However, managing single build file for compiling test code from different programming languages is a daunting task and eventually a great value add for QA companies. In our subsequent blog articles, we will be publishing more about nuances of Python automation testing.
by admin | Apr 19, 2020 | Automation Testing, Fixed, Blog |
As a software testing services company, we encounter many test automation challenges. In this blog article, we would like to list a few useful Python Test Automation Tips, With the hope that you will bookmark this article for future references.
Installing PIP on Windows
Installing PIP on Windows is a simple task and an One-time activity. When you are setting up a test bed for test automation scripts execution, you need to recollect the steps to install PIP on Windows.
Step #1 – Download get-pip.py
Step #2 – Run get-pip.py
Virtual Environments on Windows
Installing, creating and activating a Python virtual environment is pretty straight forward. On the contrary, when you play around with multiple OS, you must be aware of the fact that there are different steps involved to activate a virtual environment on Windows.
Step #1 – Install virtual environment package
Step #2 – Create a virtual environment
Note: In the above command, newenv is the environment name.
Step #3 – Activate the virtual environment. Go to the virtual environment folder on Command Prompt, navigate to ‘scripts’ folder, and run ‘activate’ command.
Creating requirements.txt
Managing the required Python packages in requirements.txt is a best practice. If you have already installed the required packages and want to create requirements.txt, then run the below command.
pip freeze > requirements.txt
Note: Once the command is executed successfully, it creates the requirements.txt with the list of installed packages.
Behave JSON Report Generation
If you are using behave BDD framework and would like to generate JSON report after automated script execution, then kick-off the behave execution using the below command.
behave -f json -o reports.json
Customized PDF Report Generation
Publishing readable and useful report is one of the important test automation success factors. You can create customized PDF report using behave JSON report. Refer the following URL for more details – Customized Test Automation PDF Report. You can create PDF report as shown below using pdf_reports package & Pug template.
Python Selenium Headless Chrome
Starting Chrome in headless mode using Selenium WebDriver can be done using ChromeOptions. Refer the below Python snippet.
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(executable_path='chromedriver.exe',chrome_options=options)
Behave Teardown
In most of the BDD frameworks, you need to write setup and tear-down in two different methods. However, in behave framework, you can write both in one method. Refer the below snippet.
from behave import fixture, use_fixture
from selenium import webdriver
@fixture
def launch_browser(context):
#Launch Browser
context.driver = webdriver.Chrome(executable_path='driverschromedriver.exe')
print("=============>Browser is launched")
yield context.driver
#Clean Up Browser
context.driver.quit()
print(context.scenario.duration)
print("=============>Browser is quit")
def before_scenario(context,scenario):
the_fixture1 = use_fixture(launch_browser, context)
Network Throttling
In Selenium WebDriver, you can set network upload & download throughput and latency to run the automation testing scripts in different network conditions.
driver.set_network_conditions(
offline=False,
latency=5, # additional latency (ms)
download_throughput=500 * 1024, # maximal throughput
upload_throughput=500 * 1024) # maximal throughput
Desktop App Automation Testing
Using pywinauto package, you can automate Desktop App Test cases.
from pywinauto.application import Application
app = Application().start("notepad.exe")
BuildBot CI
If you want to manage all CI related operations using Python code, you can try BuildBot CI.
In ConclusionAs an automation testing company, we use Python and Selenium WebDriver for multiple projects. We hope you have gained incredible insights about various test automation tips that’s has been shared in this blog article, As a continuing trend we will be keep sharing more and more useful automation testing related articles in the coming weeks.
