by admin | Nov 29, 2021 | Automation Testing, Blog, Latest Post |
We live in a world where automation is already considered to be a huge advantage now for most companies. The same goes for testing mobile and web applications. You need to have the right tools that will help you automate many of the functions of testing so you can get a higher level of test coverage and execute them at a faster rate. Let’s look at some of the tools that quality teams are looking for when testing mobile apps and web apps.
Tools for Mobile App Testing
While most web developers and testers use the Selenium framework to write tests, there are many different tools for mobile app testing. The first question to ask when choosing a tool is whether it is open-sourced or closed-source. Since most mobile apps are built with open-source frameworks, it is best to select an open-source tool, if possible.
The next question is whether the tool runs on more than one OS or just on Android or just on iOS. You will probably write tests in a language that does not depend on OS, but you should remember that Android and iOS require different test frameworks.
Not every mobile app test framework is best for every application type: native apps, web apps with a mobile interface, and hybrid apps with heavy WebView use can require different tools.
If you are running into problems deciding which tool is best for your needs, keep in mind that many companies offer free trials or free versions of their products. Just do a little bit of research before you jump in headfirst.
Tools for Web App Testing
More and more businesses rely on their web applications to streamline their operations and improve their marketing efforts. The continuous, high load and growing market expectations require web apps to undergo a variety of tests to make sure that it’s compliant with the UI standards, as well as in terms of compatibility and usability. Web app testers need to use a variety of automated software testing tools that allow them to test their products from different perspectives.
Which Testing Tool to Use?
Before making a decision on which testing tool to use for your applications, you need to take into account all the other factors that could affect how your testing will go. Some of the questions you need to consider are:
- How many tests do you need to run?
- Who will run the tests?
- How frequently do you need to run them?
- How many users do you intend to simulate?
- What scripting languages do you have experience working with?
- What platforms and browsers do you need to cover?
- Are you testing a finished product or a pre-release version?
- How important is automation for you?
- Do you have a system of tracking and analyzing the results?
- How important is a visual reporting system for you?
Conclusion
Mobile and web app testing is a crucial step in software development where companies can’t afford to cut corners. The only way to make testing much more efficient is to use the right kind of tools for the job. This guide should help you decide what kind of tools you can use for testing that will help you automate many of its functions.
If you’re in need of a QA company with a seamless track record in web and mobile testing services, Codoid is your best choice. Every new software product deserves high-quality automation testing, and our team of highly skilled QA professionals can handle any job. When it comes to QA automation services, there’s no better choice than Codoid. Partner with us today!
by admin | Nov 26, 2021 | Software Testing, Blog, Latest Post |
Prevention is better than cure is a great saying that even applies to the software development lifecycle. But it is almost impossible to develop software without any bugs on the very first try. So in order to keep up with the ever-growing need for rapid software development, it is vital for us to minimize the time taken to fix a bug. So it is very important to know the different types of software bugs detected during software testing as the complete knowledge of the type of bug you are dealing with will be instrumental in helping you understand the severity and in turn sort your priorities accordingly.
But before we explore the various types of software bugs, let’s first look into what classifies as a software bug. A software bug can be defined as an error or a fault that can render a computer program or even an entire system to crash or malfunction. So you will not get the desired or expected output despite providing a valid input. Though a glitch is different from a software bug, few might assume both to be the same. Due to the random nature of software glitches, it can be very difficult to find the cause and solve it like a software bug. But you might get lucky as glitches are known to correct themselves.
Origin of the Term ‘Bug’
Before dive in to explore the different types of software bugs, let’s take a look at some interesting trivia related to the term ‘bug’. The first-ever bug in a computer was reported by Grace Hopper back in 1946 and there is an interesting story behind how the term ‘bug’ came into use.
But some records suggest that the term ‘bug’ was used years back to denote an issue in a system.
What are the Different Types of Software Bugs?
Let’s take a list of the various types of software bugs and then explore them one by one.
- Functionality Bugs
- Performance-based Bugs
- Usability Issues
- Safety Issues
- Compatibility Issues
- System-Level Integration Bugs
- Unit Level Bugs
Functionality Bugs
Every element in an application or a website will have a purpose for its existence. At times, it is possible for a few of these elements to not function as per the need or expectations. So such errors can be termed as functional bugs. The severity of these types of bugs could also be in different ranges. To bring that into perspective, a random non-functioning button that isn’t responding to clicks is also a functionality issue, and a malfunctioning login functionality that isn’t letting the users use the software is also a functionality issue. Owing to the broad nature of this type of bug, it is one of the most common bugs you will encounter. You have to be careful and not list every single bug you find under this category. Instead, you should take a closer look and classify it accordingly.
Performance-based Bugs
But just because a function works, doesn’t mean that everything is fine. According to a report, a delay of just one second decreases the customer’s satisfaction by 16%. Now, let’s say you are at the payment gateway and have entered your password or the OTP. The payment has to be verified and a success message has to be displayed in a very short span of time. If it takes 5 minutes for the success message to be displayed, then the user would be livid with your software. Similarly, if it works smoothly only once every 5 times, you would have to be counting on your luck to get that successful message. So speed, stability, resource consumption, etc. are a few performance issues that one might face.
Usability Issues
Over the years the number of fields in the sign-up form has been on the decline and usability is one of the major reasons. If it seems like too much work, users might shy away from creating an account or using the product altogether. You might have developed a great feature that can make a task easier for the user. But if the feature is hidden away in some menu making it hard for the user to find, it most probably will remain unused. There can also be workflow bugs that can make navigation through the software very difficult.
Accessibility issues are some of the most overlooked types of usability issues that make your product inaccessible to people with disabilities. As one of the leading QA companies, we provide the best web accessibility testing services and believe that everyone should have equal access to the various applications and websites in the market. Now let’s take a look at the few common accessibility issues that are usually found.
Safety Issues
With so much of our personal information at stake, safety and privacy have become major concerns for many users. The Covid-19 pandemic also resulted in a surge of security breaches as many products were left vulnerable due to their lack of focus on security. So make sure to look out for the following security issues.
- Encryption errors
- Buffer overflows
- Vulnerabilities in XSS
- Susceptibility to SQL injections
- Weak authentication
- Loopholes in role-based access
- Insecure deserialization
- Using components with known weaknesses
- Security misconfiguration
Compatibility Issues
If we take a mobile application as an example, compatibility is one mammoth challenge that the software development team has to overcome as there are so many mobile devices that are available in the market today. Each of these devices has its own display size & resolution, operating system, processors, and so on. So if you face any issues with a certain specification, then you would have to note it down. If you are developing a website, then you have to make sure it works fine across mobile devices, tabs, and computers. So just because everything works well in the defined circumstances, you can’t expect it to work the same way in real-world scenarios that have a lot of diversity.
System-Level Integration Bugs
Though the code for the different units of the software will be written by different software developers, it must all work together seamlessly. But it is not something that happens every time and you might encounter such issues when you are testing. You might something weird happening in the UI, or you might even witness memory overflow issues because of such bugs. So you can’t directly assume it is a UI issue as it might be caused due to the inconsistencies or incompatibilities across components. This makes it very hard to locate and resolve the issue as one would have to examine a larger chunk of code to identify the problem. So make sure to be extra careful when handling such complex issues.
Unit Level Bugs
Agile teams make use of unit tests to detect issues at the early stages of the software development so that they can be spotted and fixed easily. As a prominent agile testing services provider, we always focus on unit testing in all our projects. So it is highly important for you to know about unit-level bugs. If you are unaware of what unit tests are, they are nothing but the tests that are executed once a few models of the software have been developed. For example, if the sign-up page has been successfully developed, then the testers can verify that functionality alone with maximum coverage. Beyond the functionality, one could check if the data fields are accepting inputs as per the expectations. If the password must include both upper case and lower case letters, then it should not accept a password if this condition isn’t fulfilled.
The Other Different Types of Software Bugs
We have seen the major types of software bugs that you might come across while testing. Now we will be just skimming through the few other bugs that you should know about. The code might have syntax errors that prevent the code from compiling properly. Aspects like a missing character or bracket could cause such issues. Likewise, there might be unnecessary code duplication that is just making the software more bulky and slow. Apart from that, there could be logical errors in the code that can cause loss of functionality or even crash the application. Remember the example under the unit level bugs? You might face such out-of-bound bugs outside of unit testing too.
The Severity of the Different Software Bugs
By now you would have a pretty good idea of what types of software bugs will be severe and which ones have to be prioritized. Even if there is any confusion, the following example of a few bugs in an e-commerce website will help you get a crystal clear idea.
- Critical – If there is an issue that is preventing the users from even accessing the site due to a server issue or not allowing you to place an order, then that should be termed critical as all major functionalities have been blocked.
- High-level – This might also seem like a critical bug, but it is something that reduces functionality and not something that totally destroys it. For example, if you are able to access the site and place an order but not able to track the delivery. Here the main functionality is working, but still, there is a lot of inconveniences caused to the user while performing the main functionality (Shopping).
- Medium level – Minor or less important functionalities that impact a user’s experience with the product can be categorized here. A broken link or navigation issues are perfect examples.
- Low level – Bugs in this level will most probably be UI issues like alignment, typos, color issues, and so on.
Prioritization
Conventionally, many would assume that only the critical bugs should be resolved at the earliest. Though severity plays a major role in triaging which bugs to resolve first, complexity should also be considered. Critical bugs should get the main focus irrespective of how complex it is. But along with it, even small typos or issues that can be resolved much effort should also be fixed without any delay. It doesn’t matter if the bug is severe or not, a bug is still a bug and our plan should always be to fix it at the earliest. Highly severe bugs can be prioritized based on complexity. Whereas moderately severe bugs should be fixed by the next update.
Conclusion
We hope you had a good time reading our blog and now have a pretty good idea of the different types of software bugs you might have to deal with when testing. This is a list of the most common bugs you might face, there might be scenarios where you face bugs that are more complex or different in nature. But this list is a good place for you to start and then slowly up your skills as you move forward in your testing career.
by admin | Nov 23, 2021 | Mobile App Testing, Blog, Latest Post |
Do you believe in the myth that every test case or scenario can be automated completely and effectively, and that manual testing is no longer necessary? If you don’t, that’s good news. But if you do believe in that myth, it’s time to reassess your approach towards software quality. Not all test cases or scenarios can be automated properly, and at times though automation is possible to achieve, it will not be the right path for you because of factors like ROI, repeatability, and so on. We know for a fact that manual testing is still an irreplaceable and integral part of software testing that one has to master to attain maximum quality. It has been instrumental in helping us deliver the best Mobile Testing Services to our clients. So we have curated this blog by covering the various effective manual mobile-app testing strategies we have been using over the years.
Let’s lay a roadmap first and then work around the other aspects that one should concentrate on to perform effective manual mobile-app testing. You might have the greatest tool in your hand, but it will be of much use if you don’t use it correctly. Likewise, we should know when to use manual testing and when to use automation. The very first step would be to create an effective plan, and you should know the possible resources you can use to fulfill your vision.
Develop a Plan:
You have to understand the app very well to know what tests have to be run and where automation has to be used. Here are a few pointers to keep in mind.
- What makes manual testing stand apart from automation is the manual tester who is in control. Only a real human will be able to identify the usability issues in the app. Ultimately, it is not about passing a test that sees if the app is user-friendly, it is about if the app feels user-friendly.
- Exploratory testing is another approach where the tester at the helm plays a more significant role in comparison. The intuition and domain knowledge of the tester will be able to guide them through effective tests. This type of testing is almost impossible to achieve with automation.
- Make sure to analyze your test cases and see if it has to be run only once or if it has to be run multiple times. If it has to be run just once or twice and doesn’t consume much time, there is no point in spending extra effort to automate the process.
- Localization and Internationalization make the app more accessible across the globe. Manual tests can be preferred here as when different languages are used, the UI might be affected by the change in font and word lengths.
- Black-box tests that are done during the early stages of development will also benefit from manual tests. If frequent changes are being made to the app, then implementing automation can become cumbersome due to the constant changes you’d have to make to your test scripts.
- If you are tasked with checking the compatibility of the app across different devices, then manual testing will also come in handy as there will be many test cases or scenarios that aren’t repetitive.
Types of Mobile Apps
Your test strategy must also consider the types of mobile app that is being tested. There are 3 basic categories that we would have to know about, and they are,
- Native Apps – They are applications designed specifically to work on a specific platform like iOS or Android.
- Web Apps – They are actually a responsive version of the website that can work on mobiles through a browser instead of installing the application. So they are independent of the platform.
- Hybrid Apps – As the name suggests, it is a combination of both the above types. They do have traits of a web app, but it is ultimately merged with a native app that makes it possible to be downloaded onto the phone.
Use a Wide-Range of Devices
Now let’s take a closer look at how you can combat one of the biggest challenges when it comes to testing mobile apps (i.e.) the wide range of different mobile phones that are available.
Major players in the arena have their flagship mobile launches once a year, but they also launch mid-range and budget options at more frequent intervals and these are the models that are mostly used by the people. Apart from the fresh launches, people would also be using the older versions and the app must work well with that as well.
When it comes to software, they are primarily split into iOS and Android. Android is further split into brand-specific skins like OneUI, ColorUI, and so on. Hardware options are also innumerable as Apple has started using their own M1 chips, whereas Samsung has its own line of chips (i.e.) the Exynos and also have the Snapdragon variants. Mediatek chipsets are also becoming prevalent. We could keep going if we list all the variations in the devices. Now let’s progress move on to the solution.
Cloud-Based Solutions
Purchasing every single device in the market is not a possible solution. So it is more practical to opt for cloud-based solutions like BrowserStack that gives you a wide range of real Android and iOS devices that you can test with. It is also not that complicated as you can just upload the apk or ipa file of your application and choose the device and software version you want and begin testing as though the device is with you. If you are looking for a list of the best Cloud Testing Platforms that will suit your needs, don’t miss out on our blog where we have covered this topic in-depth.
So we can basically test the application on every single device that is available on the platform. But just because we can, doesn’t mean we should. Even with the right tools, it is important to use an approach that identifies the devices you should use to test your app. Here are a few tips to keep in your mind.
- Do proper research to identify the most widely used mobile variants in the market.
- Make sure to use mobile devices with different screen resolutions and sizes.
- Identify the target audience of your app and look for data that shows the type of devices used by them.
- Your device selection should cover the various OS options available in the market. We have already discussed the different variations in Android. So make sure it is not chosen as plain Android and iOS.
- The same applies to the hardware aspects and the various features like compatibility, memory size, connectivity, and so which vary from device to device.
Emulators
A more cost-effective way would be to use emulators that mimic the actual devices instead of using the actual devices. But the reduced cost does demand additional time and effort. The reason is that you would have to download emulators of all the devices you want to test with. The other major disadvantage is the reliability issue as emulators will not be able to mimic the real-world scenarios that might happen in the real device. In fact, you won’t even be able to perform basic gestures like pinch, swipe, long press, and so on. Network operations also can’t be tested in emulators.
But as stated earlier, emulators can be used if you are on a tight budget as there are various open-source options. Since emulators will be in your local machine, they will be better in terms of speed and function without the latency that can be seen at times with cloud-based options. You could go with emulators if you have very basic testing needs or if you will use them only during the early stages of development. If not, it is better to choose one from the list.
Proper Documentation
Documentation is a very crucial aspect of software testing itself. But when it comes to manual mobile app testing, make sure you create a well-defined click path. A detailed click path doesn’t just avoid any confusion while testing, it also optimizes the time needed for each test session. Once the bug is fixed, you can refer to the documentation to recreate the scenario and find out if the bug has been fixed or not. Since every single detail matters, ensure you write down everything important you have observed while testing without leaving out any data. Documentation shouldn’t feel like the end of a conversation that is written for the sake of it. As one of the best Manual Testing Companies, we always focus on our documentation in all projects and have seen good results every single time.
Beta Testing Phase
This is pretty much an add-on phase of testing that you can use to make sure no oversight issues get through to the final release. Nowadays we have a higher number of people volunteering to beta testing programs to try out the new features of an existing app or to test a new app itself. Since the beta users will not be software testers, you will be able to get a real-world preview of how good your app is. So make sure to identify people who resemble your target audience and not random people as the results may be misleading.
Conclusion
We hope you have a clear understanding of how you can effectively test your mobile apps by taking the manual approach. There is simply no option as good as manual testing when it comes to usability and accessibility testing. So using the above-discussed strategies, we believe that you will be able to perform quality manual mobile-app testing.
by admin | Nov 19, 2021 | Software Testing, Blog, Latest Post |
Software testing is done to find anomalies in code that prohibit the product from performing as it should. These abnormalities are divided into specific groups to make debugging tasks more manageable for your team.
This article will explain the differences between bugs, errors, and issues so that QAs may be confident in cataloging, analyzing, and resolving these anomalies. Unfortunately, these flaws are guaranteed to appear while getting manual testing services for your website or app.
What is a Bug?
Any flaw in a software system that makes it act in unexpected and undesired ways is referred to as a bug. These might include unreasonable or inaccurate reactions, failed systems, and system crashes, among other things. It’s a programming fault that leads to software failure that can be discovered before the website or app is put into production.
Here are a few examples of bugs:
- Algorithm-related bugs
- Bugs that impair logic, such as infinite loops
- Uninitialized variables that cause bugs to appear
What is an Error?
Errors refer to coding or programming faults that are often caused by wrong syntax or erroneous loops. Inconsistencies or blatant fallacies in the underlying code structure cause errors to appear in the source code. Misconceptions, oversights, or misunderstandings on the developer’s part (engineers, testers, analysts, etc.) cause anomalies.
Errors come in a variety of forms:
- Calculation errors caused by incorrect formulas
- Errors in data handling due to the overwriting of required files
- Errors in configuration owing to a lack of storage capacity
What is an Issue?
The issue is a bit of a catch-all word. It signifies the presence of some abnormality in a software system that has to be resolved in software testing circles. As a result, an issue might be anything from a bug to missing or erroneous documentation, a feature request, or any other process that needs to be completed.
Issues are often assigned severity levels to be prioritized in the development and quality assurance hierarchy: high, medium, low, and cosmetic.
Clients and managers use the word “issue” to signify that something is wrong with the program at hand. It is often employed when the source of the problem is unknown. And once a problem has been identified, it must be investigated to determine its nature before being allocated to the appropriate team for remediation.
The Importance of Real Devices in Debugging and Problem-Solving
Identifying and removing any anomaly requires rigorous software testing on actual browsers and devices. Given the present state of browser-device fragmentation, every website or app must be accessible across various device-browser-OS combinations.
More than 4 billion people use various devices to access the internet, including:
- More than 9,000 different gadgets
- 21 distinct operating systems (vendor + version)
- Hundreds of browsers powered by eight primary browser engines
To ensure a favorable user experience across the board, there are 63,000 browser-platform-device combinations to test, and the number is continually growing. Due to the unique specifications of one particular browser, device, or OS, faults may appear on one combination but not on others. The program must be tested on various devices and browsers to guarantee that flaws and issues may be spotted across them.
Conclusion
Quality control operations include identifying defects, categorizing, reporting, and finally removing them. However, prevention is better than cure. The core of software quality assurance is to set up monitoring and inspection mechanisms at every level of the software development life cycle. Make sure to get optimal QA services to avoid any bug, error, or issue in your software.
Codoid is an industry leader in QA services. Our brilliant team of engineers is dedicated to helping clients overcome the challenges of complex manual testing services to grow a product. Contact us today!
by admin | Nov 19, 2021 | Performance Testing, Blog, Latest Post |
If your development is focused on providing HTTP support, Gatling is one of the best performance testing tools that we can use in load generation. Gatling is a Scala, Akka, and Netty-based open-source load and performance testing framework. Despite the fact that it is written in a domain-specific language, the tool provides us with a graphical user interface that allows us to record the scenario. When we finish recording the scenario, the GUI generates the Scala script that represents the simulation. Excited to learn how to master such a tool with so many features like the Gatling Recorder and a guide to creating a new Maven project? You’re at the right place, as we are one of the leading QA Companies with a lot of experience in using Gatling to its full potential.
Features of Gatling:
Let’s kickstart this blog by exploring the prominent features of Gatling.
Excellent HTTP Protocol – It makes Gatling a great choice for load testing web applications or directly calling APIs.
Scala simulation scripts – Since all the Gatling scripts are written in the Scala language, writing load testing scenarios in Scala provides great flexibility.
The Gatling Recorder – The Gatling recorder helps convert the flow of an application into a Gatling scenario. It is very much useful if you come across a complex web application.
The Code can be kept in the Version Control System – As the Gatling load test scenarios are written in the Scala language, they can be easily stored in the Version Control System. So the process of scenario code maintenance gets simplified.
Why Gatling?
Now that we have seen the main features of Gatling, let’s explore the other real-world benefits Gatling offers. Since it is an excellent tool for load/stress testing your system without regarding other performance requirements, several thousand concurrent users can be created from a single JVM. So, Gatling is an excellent tool for you to include in your continuous integration as it doesn’t require you to set up a distributed network of machines to perform testing. Apart from that, it is also useful if you want to write your own code rather than just recording the scripts. Now let’s see how to install Gatling.
Installing Gatling from Website
First and foremost, you have to download the Gatling performance testing tool from their official website. You will find two download options available there, one is the open-source version and the other is the Enterprise edition. We will be using the open-source version for this blog, and so you can go ahead and download that. The Enterprise edition does have a few additional features that you can explore once you are well-worse with the free version. Once your download has been completed, open the folder and unzip it.
Gatling has its own set of installation requirements for Windows and macOS. So you must have a JDK installed to execute the basic version. In addition to that, the utility requires JDK8.
Before we start, make sure to go through the Gatling manual to verify if you have got all the necessary prerequisites ready. Start by going to the bin folder in the unzipped Gatling folder to get Gatling from here. You could also do it by using the command prompt.
You could use Gatling.sh if you’re a Mac user. But since we’re on Windows, we’ll be using Gatling.bat. The tool will start up and run once you double-click on Gatling.bat. We will also be able to run a few example scripts provided by Gatling in the user directories.
If you wish to use command prompt, go to the Gatling Directory and open command prompt. Once you’ve opened the Gatling Directory Command Prompt, you can get the Gatling.bat file by using the following commands as shown in the image below.
Gatling Recorder
We have already established the fact that the Gatling recorder is one of the prominent features of Gatling. As one of the leading Test Automation Companies, we have found this feature to be very resourceful. So let’s learn how to set up and run your recorder as an HTTP proxy or a HAR converter.
The Gatling Recorder assists you in swiftly generating scenarios by acting as an HTTP proxy between the browser and the HTTP server or converting HAR (HTTP Archive) files. In either case, the recorder creates a rudimentary simulation that closely resembles your previously recorded navigation.
So, let’s take a look at how we can record using the HAR option.
Gatling Recorder Prerequisites
1) Gatling should be put in place.
2) The path to the Java Home should be set.
3) The Gatling Home path must be defined.
Gatling Recorder using the HAR File Converter
A HAR file (HTTP Archive) can be imported into the Recorder and converted into a Gatling simulation. The Chrome Developer Tools or Firebug or the NetExport Firebug add-on can be used to obtain the HAR files. Follow the below steps to convert the HAR files to Gatling simulations.
Step 1:
Right-click on the page and click on Inspect to open Developer Tools. Make sure the Preserve log option is checked under the ‘Network’ tab.
Step 2:
Once you finish navigating the website, right-click on the requests you want to export and click on the ‘Export’ option as shown in the below image.
Step 3:
Launch Gatling recorder
Use the below-mentioned command in command prompt to execute the file recorder file from the Gatling directory.
C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\bin>recorder.bat
Once you’ve run the above command, the Gatling Recorder will start and you will be able to see the following page.
Step 4:
Check the Recorder mode and make sure that the HAR Converter mode is selected. In this scenario, it is set as ‘HTTP proxy’ by default and so let’s change it to HAR Converter. We will also be seeing how to record using the ‘HTTP Proxy’ mode in the next stage of the blog.
Step 5:
Import the saved HAR file into the recorder and click on the ‘Start’ button.
Step 6:
Once it has been successfully imported, you would have to run the following script using the below command.
C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\bin>gatling.bat
Gatling Recorder using HTTP Proxy
So as stated above, we will now be seeing the steps you’ll need to follow to use the recorder using HTTP Proxy.
Step 1:
Configure the Browser
In order to use Gatling to capture our scenario, we must first configure our browser’s proxy settings. The following instructions will help you to set up Chrome for Gatling recording.
- Open the Chrome Browser.
- In the top right corner, click on the three dots to get a drop-down menu.
- Click on the ‘Settings’ option.
- Scroll down to the bottom of the settings page and click on the ‘Advanced’ drop down option.
- Click on System from the list of options that appear.
- Once you click on ‘Open your Computer Proxy Setting’, the proxy page will be shown.
- Uncheck the ‘Automatically detect settings’ option.
- Check the ‘Use the proxy server for your LAN’ option.
- The address should be “localhost” and the port should be “8000”.
- Click on the ‘Save’ Button.
Step 2:
Recording the Scenario
- First, you have to go to the bin folder of the Gatling bundle. (In our case: C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\bin)
- Double click the “recorder.bat” file or run it using command prompt. Follow the same procedure as you did when using the HAR converter.
- The recorder window will be displayed.
- Enter the port number in the local host box as per your preference. (In our case, we have used 8000).
- Enter the package name.
- Enter the class name.
- Select the following options: “Follow Redirects,” “Infer HTML resources,” “Remove Cache Headers,” and “Automatic Referrers.”
- Choose the location of the output files. We have used C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\user-files\simulations.
- Note: It’s a good idea to set simulation folders as defaults so that you wouldn’t have to copy the recorder file during load testing.
-
- Keep all the other options to be the same and don’t change any existing options as well.
- Click on the ‘Start’ button.
- Open Chrome browser and navigate the flow you wish to record.
- Once you are done, click the “Stop & Save” button to close the Gatling recording window.
- The recorded file will be saved and ready to run. The file will be stored in the directory that you specified in the previous step.
Step 3:
Executing load testing using Gatling
After the import is complete, you can run the script using the command given below.
C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\bin>gatling.bat
Creation of First Maven Project Using Scala for Gatling
Before creating a maven Project using Scala, we will look at the prerequisites you will need for its creation.
1) JDK 1.8 or Higher version should be installed in your system
2) The following environment variables have to be set
a. Java_Home Environment Variable
b. Maven_Home Environment Variable
c. Gatling_Home Environment Variable
If you want to check if you have all the prerequisites, you can simply follow the following steps.
Step 1:
Right Click on ‘This PC’ > Properties > Advanced System Settings > Environment Variables > System Variables
Now, you will find all the environment variables available in your system. If you have all the required prerequisites, you can start with the creation of your first Maven Project.
Creating Your First Project
The easiest way to create a new project is by using an “archetype”. An archetype is a general skeleton structure or template for a project.
To create the project we need to follow a few steps:
Step 1:
Run the archetype plugin using command prompt by using the following command.
mvn archetype:generate -DarchetypeGroupId=io.gatling.highcharts -DarchetypeArtifactId=gatling-highcharts-maven-archetype
Once you press enter, you will see Maven downloading many Jar Files by resolving dependencies and downloading them as needed (Only once).
Step 2:
Once all the jar files have been downloaded, assign the groupId, artifactId, and the package name for your classes before confirming the archetype creation.
Step 3:
Once you’ve provided all the required details, you will see a build success message and the location of the particular project that implies your project has been successfully created.
Step 4:
Open the Generated Project using Intellij Idea and follow the below steps to open the project using Intellij.
File > Open > Select the directory for the generated project > Click on the pom.xml file
Once you click to open the pom.xml, you will notice that it takes a little time to load all the dependencies and the project structure. It will also contain the following launchers.
- Gatling Engine
- Gatling Recorder
The Gatling load test engine can be launched by right-clicking on the Engine class in your IDE. The target/Gatling directory will be used to store the simulation reports.
You can easily start the Recorder by right-clicking on the Recorder class in your IDE. The src/test/Scala directory will be used to construct the simulations. We have attached an image reference to show how it’ll look once everything has been loaded.
Now, you are all set and ready to write your first Gatling script.
The pom file will contain the following things:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.gatling.highcharts</groupId>
<artifactId>gatling-highcharts-maven-archetype</artifactId>
<version>1</version>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<encoding>UTF-8</encoding>
<gatling.version>3.6.1</gatling.version>
<gatling-maven-plugin.version>3.1.2</gatling-maven-plugin.version>
</properties>
<dependencies>
<dependency>
<groupId>io.gatling.highcharts</groupId>
<artifactId>gatling-charts-highcharts</artifactId>
<version>${gatling.version}</version>
</dependency>
<dependency>
<groupId>io.gatling</groupId>
<artifactId>gatling-app</artifactId>
<version>${gatling.version}</version>
</dependency>
<dependency>
<groupId>io.gatling</groupId>
<artifactId>gatling-recorder</artifactId>
<version>${gatling.version}</version>
</dependency>
</dependencies>
<build>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
<version>${gatling-maven-plugin.version}</version>
</plugin>
</plugins>
</build>
</project>
Writing First Gatling Script:
To start writing your first Gatling Script first of all we need to create a package with the name simulation under SRC > test > scala as shown below.
Once you have created the simulations folder, we need to create a class under that particular folder and name it as you wish.
Now, you can start writing your script in that particular class. If you’re not aware of the general procedures to follow while writing your script, don’t be worried as we’ve got you covered.
The protocols to be followed for writing the scripts:
- The Package Name should be Mentioned
- Make sure to import all the important Gatling Classes
- The Class should extend the Simulation Class
- The script should consist of the below three integral parts
- HTTP Configuration
- Scenario
- Setup
- In the HTTP configuration part, we should provide the base URL to pass the header and the value.
- Here, we give the scenario a name and execute the request by providing the request URL and an assertion for the particular request.
- When it comes to the setup part, we must inject the number of concurrent users utilizing the setup method and pass the protocols.
The Script should be as follows:
package simulations;
import io.gatling.core.scenario.Simulation
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class TestApiSimulation extends Simulation {
//http conf
val httpConf = http.baseUrl("https://reqres.in/")
.header("Accept", value = "application/json")
.header("content type", value = "application/json")
//scenario
val scn = scenario("get user")
.exec(http("get user request")
.get("https://reqres.in/api/users/2")
.check(status is 200))
//setup
setUp(scn.inject(atOnceUsers(1))).protocols(httpConf)
}
Once the script is ready, you can run it by
- Clicking right on the ‘Engine’ option
- Click Run Engine
After you click on run Engine, you will see the script run and you will be asked to enter the description of the run in the console. Once you have entered the description, you will get the output in the console along with the report link. If you want to view the report, all you have to do is just copy the report link and paste it into any browser.
Global Information Results
Statics Report
Detail Report
Conclusion:
We hope you enjoyed reading our blog while learning how to install & use Gatling to load test an HTTP server and also how to create your own maven project. Using a graphical user interface, we can record a simulation based on a defined scenario, and once the recording is complete, we’ll be good to start our test.
by admin | Nov 16, 2021 | Exploratory Testing, Blog, Latest Post |
There are many subjective factors to consider when choosing between scripted testing and exploratory testing approaches and there is simply no universal right option. An easy way to understand this concept is to compare exploratory testing with improv in art. In certain scenarios, the artist’s talent could complement the improv method and enhance the art. But if used incorrectly in the wrong scenarios, it could degrade the art as well. So if software testing is the art, then the tester’s (artist’s) abilities and the scenario in which the exploratory testing approach is used also play a vital role. So in this blog, we will be pitting Scripted Testing vs Exploratory Testing to understand them better and use them in a way that suits your needs.
Clearing Misconceptions
As a first step, we will clear any misconceptions if there are any as few might confuse exploratory testing with ad hoc testing. We all know that scripted testing is more like clockwork where we define the test cases with every single detail that is needed for a tester to test the software without any deviations from the expectation that has been set. Yes, thinking ahead is a great way to increase efficiency, but it is not always the best method to be thorough as it is almost impossible to think of all the possible scenarios beforehand. But on the other hand, we have ad hoc testing that lacks any form of structure or goal. So exploratory testing takes the best of both concepts as it does provide the flexibility that ad hoc has to offer despite remaining structured and goal-oriented.
Scripted Testing vs Exploratory Testing
Staying on track is one area that we should pay a lot of attention to while performing exploratory testing. So let’s take a look at a few concepts that can help you not just stay focused on your goals, but also increase the efficiency of your exploratory testing. In addition to that, we will also be using these concepts to highlight how exploratory testing differs from scripted testing.
Test Charters
Test charters help you define the goals for your exploratory testing sessions. But it is different from your goals in scripted testing as the goals can be changed as you move forward with your testing. Be it scripted or exploratory testing, we always start by analyzing the information we have at our disposal. So when you find something you didn’t expect when you begin your tests, you can redefine your goals based on the new findings and stay focused on them. But when it comes to scripted testing, finding something unexpected itself is not very likely as the approach increases the tendency to see only what you want to see. You begin your software testing with the mindset of finding the bugs you already anticipate. So assigning such fluidic test charters helps you overcome this issue and prevents you from going off the rails.
Test charters should only act like a guide that doesn’t restrict the tester too much. Flying blind is never an option, but restricting yourself too much defies the purpose of exploratory testing itself. So make sure to strike the right balance between both. For example, if you are going to test a certain feature of functionality after changes have been made, then you can define boundaries based on that and test it without having to spend too much lead time that will be needed for performing scripted testing. So you can go about performing your exploratory tests by targeting the different possible scenarios, or by using your experience and expertise to develop a strategy by performing boundary value analysis & risk analysis and by following the equivalence technique. In this Scripted Testing vs Exploratory Testing blog, we will be seeing tips for better implementation under each section as well.
Tip
Classifying the bugs you find while testing or the bugs you anticipate is a great way to help create an outline for your test charter. Use your experience in testing similar software or similar concepts to curate a list of commonly found bugs and work around it. Since there is also a possibility of missing the simplest aspects when using such a method, make sure to create a checklist for the basics to avoid any oversight.
Time Boxing
We have already seen how exploratory testing helps you reduce the lead times before you start the testing process. The time gained here shouldn’t be lost elsewhere by being stagnant on a certain feature, scenario, or module. You must have an in-depth understanding of the software you are testing to create effective test charters. Likewise, that knowledge will also help you figure out how much time you’ll be needing for your tests and create test sessions. Once you start the tests, you will be able to assess the probability of finding bugs in that particular feature or module. Just because you are exploring the software doesn’t mean that you have all day. So if you are not finding any bugs, don’t spend too much time trying to find them. If you have unearthed a deep and unexpected issue that calls for more attention, modify your test session accordingly.
Such adaptability will not be available in scripted testing as you would have to go through with all the steps and tests even if there are no bugs. Yes, it will be very important to follow the scripted testing approach when you are testing critical modules or apps. But if you are in a position to test a module or software at its early stages of development, you can certainly not be able to do everything over and over again as there will be rapid changes in the code. Apart from that, as one of the most experienced Software Testing companies in USA that employ the agile methodology, we can assure you that exploratory testing is a key pillar of the agile development process.
Tip
In addition to saving time, the experience and understanding gained from exploratory testing will be instrumental in developing effective test scripts. So it is evident that you can get the best results when you combine both of these approaches in your project. Apart from time, be aware of the various metrics to gauge your performance during the process.
Divide and Conquer
Testing the application in its entirety is not a good option even if it is a basic application with a fewer number of modules. You’ll lose track of your goals and objectives when testing in such a random manner. So, add structure to your exploratory testing by dividing the software into various modules. So in terms of the approach, exploratory testing is not very much different from what happens with scripted testing in this case. Once you check each module off your list, you will have more insight into the bugs you may find in the application. Ensure you evaluate your original plan based on the results you have in your hand. If you identify any common issue across all the modules, change your plan accordingly. You would also have to look for the root cause of the bugs in each module and improve your test charters with better predictions.
Tip
You can use mind maps to easily identify the root cause and common issues across the various modules. Mind maps will provide you with a visual representation that will help you connect the unseen dots. In addition to that, exploratory testing does require proper documentation for it to be effective and mind maps will come in handy to easily record the issues and evaluate them later on. So for a better understanding of how effective mind maps can be for documentation during exploratory testing, visit our blog.
Exploratory Testing Techniques to Master
Though the individual skills of the software testers who perform both these types of software testing are important, the skills do play a more vital role when it comes to exploratory testing. So if you are focused on improving your exploratory testing skills, then make sure to read our blog about ‘The 5 Exploratory Testing Techniques to Master’. If not, here is the list of the 5 techniques we have explored in that blog.
- Exploring User Stories
- Being Bold Enough to Test New Waters
- Getting handy with Mind Maps
- Improving Your Integration Testing Skills
- Following an End-to-End Approach
By now you would have clearly understood that the experience of the QA performing the exploratory testing is very important. Experience alone isn’t enough as only dedicated work in those years of experience will equip you with the required expertise to do effective exploratory testing. Experience is something that you can’t get overnight, but the above-mentioned techniques will definitely help your cause.
Conclusion
So to sum things up in this Scripted Testing vs Exploratory Testing blog, exploratory testing is a great option when you want to test during the early stages of software development. Since exploratory testing is a context-driven approach, it will also be a great choice to use when following the agile method of software development as it helps keep up with the short scrum cycles. It is also a great way to identify usability issues in comparison to scripted testing methods. But exploratory testing alone is not enough by any means. As one of the best Automation Testing Companies, we also focus on scripted testing as automation is easier to implement in this approach. You can also avoid any oversight issues or bias issues by following the scripted testing approach. So we would highly recommend you to use both these methods for the best results.
