by admin | Nov 23, 2021 | Mobile App Testing, Blog, Latest Post |
Do you believe in the myth that every test case or scenario can be automated completely and effectively, and that manual testing is no longer necessary? If you don’t, that’s good news. But if you do believe in that myth, it’s time to reassess your approach towards software quality. Not all test cases or scenarios can be automated properly, and at times though automation is possible to achieve, it will not be the right path for you because of factors like ROI, repeatability, and so on. We know for a fact that manual testing is still an irreplaceable and integral part of software testing that one has to master to attain maximum quality. It has been instrumental in helping us deliver the best Mobile Testing Services to our clients. So we have curated this blog by covering the various effective manual mobile-app testing strategies we have been using over the years.
Let’s lay a roadmap first and then work around the other aspects that one should concentrate on to perform effective manual mobile-app testing. You might have the greatest tool in your hand, but it will be of much use if you don’t use it correctly. Likewise, we should know when to use manual testing and when to use automation. The very first step would be to create an effective plan, and you should know the possible resources you can use to fulfill your vision.
Develop a Plan:
You have to understand the app very well to know what tests have to be run and where automation has to be used. Here are a few pointers to keep in mind.
- What makes manual testing stand apart from automation is the manual tester who is in control. Only a real human will be able to identify the usability issues in the app. Ultimately, it is not about passing a test that sees if the app is user-friendly, it is about if the app feels user-friendly.
- Exploratory testing is another approach where the tester at the helm plays a more significant role in comparison. The intuition and domain knowledge of the tester will be able to guide them through effective tests. This type of testing is almost impossible to achieve with automation.
- Make sure to analyze your test cases and see if it has to be run only once or if it has to be run multiple times. If it has to be run just once or twice and doesn’t consume much time, there is no point in spending extra effort to automate the process.
- Localization and Internationalization make the app more accessible across the globe. Manual tests can be preferred here as when different languages are used, the UI might be affected by the change in font and word lengths.
- Black-box tests that are done during the early stages of development will also benefit from manual tests. If frequent changes are being made to the app, then implementing automation can become cumbersome due to the constant changes you’d have to make to your test scripts.
- If you are tasked with checking the compatibility of the app across different devices, then manual testing will also come in handy as there will be many test cases or scenarios that aren’t repetitive.
Types of Mobile Apps
Your test strategy must also consider the types of mobile app that is being tested. There are 3 basic categories that we would have to know about, and they are,
- Native Apps – They are applications designed specifically to work on a specific platform like iOS or Android.
- Web Apps – They are actually a responsive version of the website that can work on mobiles through a browser instead of installing the application. So they are independent of the platform.
- Hybrid Apps – As the name suggests, it is a combination of both the above types. They do have traits of a web app, but it is ultimately merged with a native app that makes it possible to be downloaded onto the phone.
Use a Wide-Range of Devices
Now let’s take a closer look at how you can combat one of the biggest challenges when it comes to testing mobile apps (i.e.) the wide range of different mobile phones that are available.
Major players in the arena have their flagship mobile launches once a year, but they also launch mid-range and budget options at more frequent intervals and these are the models that are mostly used by the people. Apart from the fresh launches, people would also be using the older versions and the app must work well with that as well.
When it comes to software, they are primarily split into iOS and Android. Android is further split into brand-specific skins like OneUI, ColorUI, and so on. Hardware options are also innumerable as Apple has started using their own M1 chips, whereas Samsung has its own line of chips (i.e.) the Exynos and also have the Snapdragon variants. Mediatek chipsets are also becoming prevalent. We could keep going if we list all the variations in the devices. Now let’s progress move on to the solution.
Cloud-Based Solutions
Purchasing every single device in the market is not a possible solution. So it is more practical to opt for cloud-based solutions like BrowserStack that gives you a wide range of real Android and iOS devices that you can test with. It is also not that complicated as you can just upload the apk or ipa file of your application and choose the device and software version you want and begin testing as though the device is with you. If you are looking for a list of the best Cloud Testing Platforms that will suit your needs, don’t miss out on our blog where we have covered this topic in-depth.
So we can basically test the application on every single device that is available on the platform. But just because we can, doesn’t mean we should. Even with the right tools, it is important to use an approach that identifies the devices you should use to test your app. Here are a few tips to keep in your mind.
- Do proper research to identify the most widely used mobile variants in the market.
- Make sure to use mobile devices with different screen resolutions and sizes.
- Identify the target audience of your app and look for data that shows the type of devices used by them.
- Your device selection should cover the various OS options available in the market. We have already discussed the different variations in Android. So make sure it is not chosen as plain Android and iOS.
- The same applies to the hardware aspects and the various features like compatibility, memory size, connectivity, and so which vary from device to device.
Emulators
A more cost-effective way would be to use emulators that mimic the actual devices instead of using the actual devices. But the reduced cost does demand additional time and effort. The reason is that you would have to download emulators of all the devices you want to test with. The other major disadvantage is the reliability issue as emulators will not be able to mimic the real-world scenarios that might happen in the real device. In fact, you won’t even be able to perform basic gestures like pinch, swipe, long press, and so on. Network operations also can’t be tested in emulators.
But as stated earlier, emulators can be used if you are on a tight budget as there are various open-source options. Since emulators will be in your local machine, they will be better in terms of speed and function without the latency that can be seen at times with cloud-based options. You could go with emulators if you have very basic testing needs or if you will use them only during the early stages of development. If not, it is better to choose one from the list.
Proper Documentation
Documentation is a very crucial aspect of software testing itself. But when it comes to manual mobile app testing, make sure you create a well-defined click path. A detailed click path doesn’t just avoid any confusion while testing, it also optimizes the time needed for each test session. Once the bug is fixed, you can refer to the documentation to recreate the scenario and find out if the bug has been fixed or not. Since every single detail matters, ensure you write down everything important you have observed while testing without leaving out any data. Documentation shouldn’t feel like the end of a conversation that is written for the sake of it. As one of the best Manual Testing Companies, we always focus on our documentation in all projects and have seen good results every single time.
Beta Testing Phase
This is pretty much an add-on phase of testing that you can use to make sure no oversight issues get through to the final release. Nowadays we have a higher number of people volunteering to beta testing programs to try out the new features of an existing app or to test a new app itself. Since the beta users will not be software testers, you will be able to get a real-world preview of how good your app is. So make sure to identify people who resemble your target audience and not random people as the results may be misleading.
Conclusion
We hope you have a clear understanding of how you can effectively test your mobile apps by taking the manual approach. There is simply no option as good as manual testing when it comes to usability and accessibility testing. So using the above-discussed strategies, we believe that you will be able to perform quality manual mobile-app testing.
by admin | Nov 19, 2021 | Software Testing, Blog, Latest Post |
Software testing is done to find anomalies in code that prohibit the product from performing as it should. These abnormalities are divided into specific groups to make debugging tasks more manageable for your team.
This article will explain the differences between bugs, errors, and issues so that QAs may be confident in cataloging, analyzing, and resolving these anomalies. Unfortunately, these flaws are guaranteed to appear while getting manual testing services for your website or app.
What is a Bug?
Any flaw in a software system that makes it act in unexpected and undesired ways is referred to as a bug. These might include unreasonable or inaccurate reactions, failed systems, and system crashes, among other things. It’s a programming fault that leads to software failure that can be discovered before the website or app is put into production.
Here are a few examples of bugs:
- Algorithm-related bugs
- Bugs that impair logic, such as infinite loops
- Uninitialized variables that cause bugs to appear
What is an Error?
Errors refer to coding or programming faults that are often caused by wrong syntax or erroneous loops. Inconsistencies or blatant fallacies in the underlying code structure cause errors to appear in the source code. Misconceptions, oversights, or misunderstandings on the developer’s part (engineers, testers, analysts, etc.) cause anomalies.
Errors come in a variety of forms:
- Calculation errors caused by incorrect formulas
- Errors in data handling due to the overwriting of required files
- Errors in configuration owing to a lack of storage capacity
What is an Issue?
The issue is a bit of a catch-all word. It signifies the presence of some abnormality in a software system that has to be resolved in software testing circles. As a result, an issue might be anything from a bug to missing or erroneous documentation, a feature request, or any other process that needs to be completed.
Issues are often assigned severity levels to be prioritized in the development and quality assurance hierarchy: high, medium, low, and cosmetic.
Clients and managers use the word “issue” to signify that something is wrong with the program at hand. It is often employed when the source of the problem is unknown. And once a problem has been identified, it must be investigated to determine its nature before being allocated to the appropriate team for remediation.
The Importance of Real Devices in Debugging and Problem-Solving
Identifying and removing any anomaly requires rigorous software testing on actual browsers and devices. Given the present state of browser-device fragmentation, every website or app must be accessible across various device-browser-OS combinations.
More than 4 billion people use various devices to access the internet, including:
- More than 9,000 different gadgets
- 21 distinct operating systems (vendor + version)
- Hundreds of browsers powered by eight primary browser engines
To ensure a favorable user experience across the board, there are 63,000 browser-platform-device combinations to test, and the number is continually growing. Due to the unique specifications of one particular browser, device, or OS, faults may appear on one combination but not on others. The program must be tested on various devices and browsers to guarantee that flaws and issues may be spotted across them.
Conclusion
Quality control operations include identifying defects, categorizing, reporting, and finally removing them. However, prevention is better than cure. The core of software quality assurance is to set up monitoring and inspection mechanisms at every level of the software development life cycle. Make sure to get optimal QA services to avoid any bug, error, or issue in your software.
Codoid is an industry leader in QA services. Our brilliant team of engineers is dedicated to helping clients overcome the challenges of complex manual testing services to grow a product. Contact us today!
by admin | Nov 19, 2021 | Performance Testing, Blog, Latest Post |
If your development is focused on providing HTTP support, Gatling is one of the best performance testing tools that we can use in load generation. Gatling is a Scala, Akka, and Netty-based open-source load and performance testing framework. Despite the fact that it is written in a domain-specific language, the tool provides us with a graphical user interface that allows us to record the scenario. When we finish recording the scenario, the GUI generates the Scala script that represents the simulation. Excited to learn how to master such a tool with so many features like the Gatling Recorder and a guide to creating a new Maven project? You’re at the right place, as we are one of the leading QA Companies with a lot of experience in using Gatling to its full potential.
Features of Gatling:
Let’s kickstart this blog by exploring the prominent features of Gatling.
Excellent HTTP Protocol – It makes Gatling a great choice for load testing web applications or directly calling APIs.
Scala simulation scripts – Since all the Gatling scripts are written in the Scala language, writing load testing scenarios in Scala provides great flexibility.
The Gatling Recorder – The Gatling recorder helps convert the flow of an application into a Gatling scenario. It is very much useful if you come across a complex web application.
The Code can be kept in the Version Control System – As the Gatling load test scenarios are written in the Scala language, they can be easily stored in the Version Control System. So the process of scenario code maintenance gets simplified.
Why Gatling?
Now that we have seen the main features of Gatling, let’s explore the other real-world benefits Gatling offers. Since it is an excellent tool for load/stress testing your system without regarding other performance requirements, several thousand concurrent users can be created from a single JVM. So, Gatling is an excellent tool for you to include in your continuous integration as it doesn’t require you to set up a distributed network of machines to perform testing. Apart from that, it is also useful if you want to write your own code rather than just recording the scripts. Now let’s see how to install Gatling.
Installing Gatling from Website
First and foremost, you have to download the Gatling performance testing tool from their official website. You will find two download options available there, one is the open-source version and the other is the Enterprise edition. We will be using the open-source version for this blog, and so you can go ahead and download that. The Enterprise edition does have a few additional features that you can explore once you are well-worse with the free version. Once your download has been completed, open the folder and unzip it.
Gatling has its own set of installation requirements for Windows and macOS. So you must have a JDK installed to execute the basic version. In addition to that, the utility requires JDK8.
Before we start, make sure to go through the Gatling manual to verify if you have got all the necessary prerequisites ready. Start by going to the bin folder in the unzipped Gatling folder to get Gatling from here. You could also do it by using the command prompt.
You could use Gatling.sh if you’re a Mac user. But since we’re on Windows, we’ll be using Gatling.bat. The tool will start up and run once you double-click on Gatling.bat. We will also be able to run a few example scripts provided by Gatling in the user directories.
If you wish to use command prompt, go to the Gatling Directory and open command prompt. Once you’ve opened the Gatling Directory Command Prompt, you can get the Gatling.bat file by using the following commands as shown in the image below.
Gatling Recorder
We have already established the fact that the Gatling recorder is one of the prominent features of Gatling. As one of the leading Test Automation Companies, we have found this feature to be very resourceful. So let’s learn how to set up and run your recorder as an HTTP proxy or a HAR converter.
The Gatling Recorder assists you in swiftly generating scenarios by acting as an HTTP proxy between the browser and the HTTP server or converting HAR (HTTP Archive) files. In either case, the recorder creates a rudimentary simulation that closely resembles your previously recorded navigation.
So, let’s take a look at how we can record using the HAR option.
Gatling Recorder Prerequisites
1) Gatling should be put in place.
2) The path to the Java Home should be set.
3) The Gatling Home path must be defined.
Gatling Recorder using the HAR File Converter
A HAR file (HTTP Archive) can be imported into the Recorder and converted into a Gatling simulation. The Chrome Developer Tools or Firebug or the NetExport Firebug add-on can be used to obtain the HAR files. Follow the below steps to convert the HAR files to Gatling simulations.
Step 1:
Right-click on the page and click on Inspect to open Developer Tools. Make sure the Preserve log option is checked under the ‘Network’ tab.
Step 2:
Once you finish navigating the website, right-click on the requests you want to export and click on the ‘Export’ option as shown in the below image.
Step 3:
Launch Gatling recorder
Use the below-mentioned command in command prompt to execute the file recorder file from the Gatling directory.
C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\bin>recorder.bat
Once you’ve run the above command, the Gatling Recorder will start and you will be able to see the following page.
Step 4:
Check the Recorder mode and make sure that the HAR Converter mode is selected. In this scenario, it is set as ‘HTTP proxy’ by default and so let’s change it to HAR Converter. We will also be seeing how to record using the ‘HTTP Proxy’ mode in the next stage of the blog.
Step 5:
Import the saved HAR file into the recorder and click on the ‘Start’ button.
Step 6:
Once it has been successfully imported, you would have to run the following script using the below command.
C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\bin>gatling.bat
Gatling Recorder using HTTP Proxy
So as stated above, we will now be seeing the steps you’ll need to follow to use the recorder using HTTP Proxy.
Step 1:
Configure the Browser
In order to use Gatling to capture our scenario, we must first configure our browser’s proxy settings. The following instructions will help you to set up Chrome for Gatling recording.
- Open the Chrome Browser.
- In the top right corner, click on the three dots to get a drop-down menu.
- Click on the ‘Settings’ option.
- Scroll down to the bottom of the settings page and click on the ‘Advanced’ drop down option.
- Click on System from the list of options that appear.
- Once you click on ‘Open your Computer Proxy Setting’, the proxy page will be shown.
- Uncheck the ‘Automatically detect settings’ option.
- Check the ‘Use the proxy server for your LAN’ option.
- The address should be “localhost” and the port should be “8000”.
- Click on the ‘Save’ Button.
Step 2:
Recording the Scenario
- First, you have to go to the bin folder of the Gatling bundle. (In our case: C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\bin)
- Double click the “recorder.bat” file or run it using command prompt. Follow the same procedure as you did when using the HAR converter.
- The recorder window will be displayed.
- Enter the port number in the local host box as per your preference. (In our case, we have used 8000).
- Enter the package name.
- Enter the class name.
- Select the following options: “Follow Redirects,” “Infer HTML resources,” “Remove Cache Headers,” and “Automatic Referrers.”
- Choose the location of the output files. We have used C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\user-files\simulations.
- Note: It’s a good idea to set simulation folders as defaults so that you wouldn’t have to copy the recorder file during load testing.
-
- Keep all the other options to be the same and don’t change any existing options as well.
- Click on the ‘Start’ button.
- Open Chrome browser and navigate the flow you wish to record.
- Once you are done, click the “Stop & Save” button to close the Gatling recording window.
- The recorded file will be saved and ready to run. The file will be stored in the directory that you specified in the previous step.
Step 3:
Executing load testing using Gatling
After the import is complete, you can run the script using the command given below.
C:\Gatling\gatling-charts-highcharts-bundle-3.6.1\bin>gatling.bat
Creation of First Maven Project Using Scala for Gatling
Before creating a maven Project using Scala, we will look at the prerequisites you will need for its creation.
1) JDK 1.8 or Higher version should be installed in your system
2) The following environment variables have to be set
a. Java_Home Environment Variable
b. Maven_Home Environment Variable
c. Gatling_Home Environment Variable
If you want to check if you have all the prerequisites, you can simply follow the following steps.
Step 1:
Right Click on ‘This PC’ > Properties > Advanced System Settings > Environment Variables > System Variables
Now, you will find all the environment variables available in your system. If you have all the required prerequisites, you can start with the creation of your first Maven Project.
Creating Your First Project
The easiest way to create a new project is by using an “archetype”. An archetype is a general skeleton structure or template for a project.
To create the project we need to follow a few steps:
Step 1:
Run the archetype plugin using command prompt by using the following command.
mvn archetype:generate -DarchetypeGroupId=io.gatling.highcharts -DarchetypeArtifactId=gatling-highcharts-maven-archetype
Once you press enter, you will see Maven downloading many Jar Files by resolving dependencies and downloading them as needed (Only once).
Step 2:
Once all the jar files have been downloaded, assign the groupId, artifactId, and the package name for your classes before confirming the archetype creation.
Step 3:
Once you’ve provided all the required details, you will see a build success message and the location of the particular project that implies your project has been successfully created.
Step 4:
Open the Generated Project using Intellij Idea and follow the below steps to open the project using Intellij.
File > Open > Select the directory for the generated project > Click on the pom.xml file
Once you click to open the pom.xml, you will notice that it takes a little time to load all the dependencies and the project structure. It will also contain the following launchers.
- Gatling Engine
- Gatling Recorder
The Gatling load test engine can be launched by right-clicking on the Engine class in your IDE. The target/Gatling directory will be used to store the simulation reports.
You can easily start the Recorder by right-clicking on the Recorder class in your IDE. The src/test/Scala directory will be used to construct the simulations. We have attached an image reference to show how it’ll look once everything has been loaded.
Now, you are all set and ready to write your first Gatling script.
The pom file will contain the following things:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.gatling.highcharts</groupId>
<artifactId>gatling-highcharts-maven-archetype</artifactId>
<version>1</version>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<encoding>UTF-8</encoding>
<gatling.version>3.6.1</gatling.version>
<gatling-maven-plugin.version>3.1.2</gatling-maven-plugin.version>
</properties>
<dependencies>
<dependency>
<groupId>io.gatling.highcharts</groupId>
<artifactId>gatling-charts-highcharts</artifactId>
<version>${gatling.version}</version>
</dependency>
<dependency>
<groupId>io.gatling</groupId>
<artifactId>gatling-app</artifactId>
<version>${gatling.version}</version>
</dependency>
<dependency>
<groupId>io.gatling</groupId>
<artifactId>gatling-recorder</artifactId>
<version>${gatling.version}</version>
</dependency>
</dependencies>
<build>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
<version>${gatling-maven-plugin.version}</version>
</plugin>
</plugins>
</build>
</project>
Writing First Gatling Script:
To start writing your first Gatling Script first of all we need to create a package with the name simulation under SRC > test > scala as shown below.
Once you have created the simulations folder, we need to create a class under that particular folder and name it as you wish.
Now, you can start writing your script in that particular class. If you’re not aware of the general procedures to follow while writing your script, don’t be worried as we’ve got you covered.
The protocols to be followed for writing the scripts:
- The Package Name should be Mentioned
- Make sure to import all the important Gatling Classes
- The Class should extend the Simulation Class
- The script should consist of the below three integral parts
- HTTP Configuration
- Scenario
- Setup
- In the HTTP configuration part, we should provide the base URL to pass the header and the value.
- Here, we give the scenario a name and execute the request by providing the request URL and an assertion for the particular request.
- When it comes to the setup part, we must inject the number of concurrent users utilizing the setup method and pass the protocols.
The Script should be as follows:
package simulations;
import io.gatling.core.scenario.Simulation
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class TestApiSimulation extends Simulation {
//http conf
val httpConf = http.baseUrl("https://reqres.in/")
.header("Accept", value = "application/json")
.header("content type", value = "application/json")
//scenario
val scn = scenario("get user")
.exec(http("get user request")
.get("https://reqres.in/api/users/2")
.check(status is 200))
//setup
setUp(scn.inject(atOnceUsers(1))).protocols(httpConf)
}
Once the script is ready, you can run it by
- Clicking right on the ‘Engine’ option
- Click Run Engine
After you click on run Engine, you will see the script run and you will be asked to enter the description of the run in the console. Once you have entered the description, you will get the output in the console along with the report link. If you want to view the report, all you have to do is just copy the report link and paste it into any browser.
Global Information Results
Statics Report
Detail Report
Conclusion:
We hope you enjoyed reading our blog while learning how to install & use Gatling to load test an HTTP server and also how to create your own maven project. Using a graphical user interface, we can record a simulation based on a defined scenario, and once the recording is complete, we’ll be good to start our test.
by admin | Nov 16, 2021 | Exploratory Testing, Blog, Latest Post |
There are many subjective factors to consider when choosing between scripted testing and exploratory testing approaches and there is simply no universal right option. An easy way to understand this concept is to compare exploratory testing with improv in art. In certain scenarios, the artist’s talent could complement the improv method and enhance the art. But if used incorrectly in the wrong scenarios, it could degrade the art as well. So if software testing is the art, then the tester’s (artist’s) abilities and the scenario in which the exploratory testing approach is used also play a vital role. So in this blog, we will be pitting Scripted Testing vs Exploratory Testing to understand them better and use them in a way that suits your needs.
Clearing Misconceptions
As a first step, we will clear any misconceptions if there are any as few might confuse exploratory testing with ad hoc testing. We all know that scripted testing is more like clockwork where we define the test cases with every single detail that is needed for a tester to test the software without any deviations from the expectation that has been set. Yes, thinking ahead is a great way to increase efficiency, but it is not always the best method to be thorough as it is almost impossible to think of all the possible scenarios beforehand. But on the other hand, we have ad hoc testing that lacks any form of structure or goal. So exploratory testing takes the best of both concepts as it does provide the flexibility that ad hoc has to offer despite remaining structured and goal-oriented.
Scripted Testing vs Exploratory Testing
Staying on track is one area that we should pay a lot of attention to while performing exploratory testing. So let’s take a look at a few concepts that can help you not just stay focused on your goals, but also increase the efficiency of your exploratory testing. In addition to that, we will also be using these concepts to highlight how exploratory testing differs from scripted testing.
Test Charters
Test charters help you define the goals for your exploratory testing sessions. But it is different from your goals in scripted testing as the goals can be changed as you move forward with your testing. Be it scripted or exploratory testing, we always start by analyzing the information we have at our disposal. So when you find something you didn’t expect when you begin your tests, you can redefine your goals based on the new findings and stay focused on them. But when it comes to scripted testing, finding something unexpected itself is not very likely as the approach increases the tendency to see only what you want to see. You begin your software testing with the mindset of finding the bugs you already anticipate. So assigning such fluidic test charters helps you overcome this issue and prevents you from going off the rails.
Test charters should only act like a guide that doesn’t restrict the tester too much. Flying blind is never an option, but restricting yourself too much defies the purpose of exploratory testing itself. So make sure to strike the right balance between both. For example, if you are going to test a certain feature of functionality after changes have been made, then you can define boundaries based on that and test it without having to spend too much lead time that will be needed for performing scripted testing. So you can go about performing your exploratory tests by targeting the different possible scenarios, or by using your experience and expertise to develop a strategy by performing boundary value analysis & risk analysis and by following the equivalence technique. In this Scripted Testing vs Exploratory Testing blog, we will be seeing tips for better implementation under each section as well.
Tip
Classifying the bugs you find while testing or the bugs you anticipate is a great way to help create an outline for your test charter. Use your experience in testing similar software or similar concepts to curate a list of commonly found bugs and work around it. Since there is also a possibility of missing the simplest aspects when using such a method, make sure to create a checklist for the basics to avoid any oversight.
Time Boxing
We have already seen how exploratory testing helps you reduce the lead times before you start the testing process. The time gained here shouldn’t be lost elsewhere by being stagnant on a certain feature, scenario, or module. You must have an in-depth understanding of the software you are testing to create effective test charters. Likewise, that knowledge will also help you figure out how much time you’ll be needing for your tests and create test sessions. Once you start the tests, you will be able to assess the probability of finding bugs in that particular feature or module. Just because you are exploring the software doesn’t mean that you have all day. So if you are not finding any bugs, don’t spend too much time trying to find them. If you have unearthed a deep and unexpected issue that calls for more attention, modify your test session accordingly.
Such adaptability will not be available in scripted testing as you would have to go through with all the steps and tests even if there are no bugs. Yes, it will be very important to follow the scripted testing approach when you are testing critical modules or apps. But if you are in a position to test a module or software at its early stages of development, you can certainly not be able to do everything over and over again as there will be rapid changes in the code. Apart from that, as one of the most experienced Software Testing companies in USA that employ the agile methodology, we can assure you that exploratory testing is a key pillar of the agile development process.
Tip
In addition to saving time, the experience and understanding gained from exploratory testing will be instrumental in developing effective test scripts. So it is evident that you can get the best results when you combine both of these approaches in your project. Apart from time, be aware of the various metrics to gauge your performance during the process.
Divide and Conquer
Testing the application in its entirety is not a good option even if it is a basic application with a fewer number of modules. You’ll lose track of your goals and objectives when testing in such a random manner. So, add structure to your exploratory testing by dividing the software into various modules. So in terms of the approach, exploratory testing is not very much different from what happens with scripted testing in this case. Once you check each module off your list, you will have more insight into the bugs you may find in the application. Ensure you evaluate your original plan based on the results you have in your hand. If you identify any common issue across all the modules, change your plan accordingly. You would also have to look for the root cause of the bugs in each module and improve your test charters with better predictions.
Tip
You can use mind maps to easily identify the root cause and common issues across the various modules. Mind maps will provide you with a visual representation that will help you connect the unseen dots. In addition to that, exploratory testing does require proper documentation for it to be effective and mind maps will come in handy to easily record the issues and evaluate them later on. So for a better understanding of how effective mind maps can be for documentation during exploratory testing, visit our blog.
Exploratory Testing Techniques to Master
Though the individual skills of the software testers who perform both these types of software testing are important, the skills do play a more vital role when it comes to exploratory testing. So if you are focused on improving your exploratory testing skills, then make sure to read our blog about ‘The 5 Exploratory Testing Techniques to Master’. If not, here is the list of the 5 techniques we have explored in that blog.
- Exploring User Stories
- Being Bold Enough to Test New Waters
- Getting handy with Mind Maps
- Improving Your Integration Testing Skills
- Following an End-to-End Approach
By now you would have clearly understood that the experience of the QA performing the exploratory testing is very important. Experience alone isn’t enough as only dedicated work in those years of experience will equip you with the required expertise to do effective exploratory testing. Experience is something that you can’t get overnight, but the above-mentioned techniques will definitely help your cause.
Conclusion
So to sum things up in this Scripted Testing vs Exploratory Testing blog, exploratory testing is a great option when you want to test during the early stages of software development. Since exploratory testing is a context-driven approach, it will also be a great choice to use when following the agile method of software development as it helps keep up with the short scrum cycles. It is also a great way to identify usability issues in comparison to scripted testing methods. But exploratory testing alone is not enough by any means. As one of the best Automation Testing Companies, we also focus on scripted testing as automation is easier to implement in this approach. You can also avoid any oversight issues or bias issues by following the scripted testing approach. So we would highly recommend you to use both these methods for the best results.
by admin | Nov 12, 2021 | Mobile App Testing, Blog, Latest Post |
Today’s trend in technology is the use of mobile devices. The reason for this is because a smartphone can do the same things a computer can. This is why many companies resort to developing mobile applications to expand their operations. While it’s a sensible choice, not everyone succeeds in this endeavor.
A mobile application may have a successful launch, but its maintenance can make or break it. For this reason, proper app testing must be conducted first to set the parameters of how an app will be maintained.
In this article, we list down eight reasons app testing is vital to the mobile app development process. Read on below to learn more.
#1 – Implements Quality Assurance
Similar to most products, quality assurance applies to mobile apps too. After all, you don’t make a mobile app and rush immediately to launch—you’ll want to make sure that everything is working properly for it to succeed.
#2 – Helps with Revenue Generation
There’s no doubt that the growing need for smartphones only means that apps must be well-developed. For companies, this only means one thing: to capitalize on mobile apps and generate more revenue.
App testing will assist you in developing strategies to allocate resources for activities such as branding, development, and marketing up to the final launch of the app. When done right, you will maximize your profit.
#3 – Improves User Interface
An excellent user interface can bring in more users, so it has to be user-friendly. Some pointers to take note of when making a user interface design are:
- Easy navigation
- Fast load times
- Includes a search feature
- Simple design
As long as these four are present, your user interface will be good, making it easier to use your app.
#4 – Guides with Update Releases
You might think that your app is already perfect, and there’s nothing wrong with that. However, there will come a time when the app’s functionality is considered outdated. For this reason, updates to maintain the app’s functionality will be released.
It’s not even just functionality because other improvements are also included with updates, such as increased security. App testing ensures that updates will work for all devices, and it will also give you time to evaluate what can be improved.
#5 – Reduces Risks for Your Business
It’s estimated that mobile apps from more prominent companies cost somewhere between $100,000 to $500,000 to develop. With that amount of money, you definitely don’t want it to go to waste.
Through app testing, you will know the risks that will pose a threat to your mobile app. These issues are mitigated because you can detect problems earlier.
#6 – Helps Establish Your Brand
Many businesses functioning through apps must always look for ways that will distinguish them from the competition. App testing will help implement recommendations and feedback from test users, allowing more users to enjoy using the app.
#7 – Attracts More Users
If your mobile app is tested, it’s a guarantee that they’ll use it for their comfort. With a solid user base, you can attract more users to your app whenever your loyal users recommend the app to people they know. Referrals trying out your app because of its positive reputation is an added bonus.
#8 – Ensures Compatibility Across Various Devices
The number of smartphones being manufactured each year is rising. Because of this, compatibility between devices must be prioritized. App testing ensures that the app can function without issues regardless of whether the device is Android or Apple.
Conclusion
App testing is a critical part of the mobile app development process that must not be skipped or rushed in any way. If you don’t conduct app testing, you’re setting yourself up for failure because it won’t do users any good, leading you to lose business.
Codoid offers top-quality mobile app testing services to make sure that your mobile app functions correctly. A mobile app must always help solve a user’s needs, and an incomplete app won’t do that. Contact us today to learn more!
by admin | Nov 12, 2021 | Automation Testing, Blog, Latest Post |
Robot Framework is an open-source Python-based automation framework used for Robotic Process Automation (RPA) and also for Acceptance Testing & Acceptances Test-Driven Development (ATDD). It is used with the keyword-driven testing approach for which testers can write automation test scripts easily. The Test cases are written using the keyword style in a tabular format making them easy to understand. Robot Framework does provide good support for external libraries. But Selenium library is the most popular library used with Robot Framework for web development and UI testing. As a leading mobile app testing service provider, we have been using Robot Framework in our various Android and iOS app testing projects. So we have written this end-to-end Robot Framework Tutorial that covers everything one has to know.
The Basic requirements for robot framework
Since the Robot framework is built based on Python, we have to install the latest version of Python by downloading it from their official site. After successful installation, set up the Path environment variable in your local system.
Steps to Install Robot Framework:-
1. You can check if the Python setup has been installed properly by opening Command Prompt and typing the below command.
2. Use the below command to check the pip version. (Pip is a Python package manager)
3. Now we’re all set to install the Robot framework
a. For a straightforward installation of the robot framework, you can use the command
pip install robotframework
b. If there is a need to install the robot framework without any cache, you can use this code instead.
pip install --no-cache-dir robotframework
c. If for any reason, you want to uninstall the Robot framework, then you can do so by using the below code.
pip uninstall robotframework
4. Just like how we verified Java, we can use the below commands to check it using Command Prompt.
(i)
This command will show the details of the robot framework and the framework location.
(ii.)
Once we enter this command, the desired result that should be displayed is “No broken requirements found.” If any other message apart from this (i.e.) a warning is displayed, we need to check the warning related to the setup and install it properly.
We can also verify if the Robot Framework has also been installed properly in our Local system.
Go to the Python folder > Lib > site-packages > verify if Robot Framework is installed.
5. To verify the version of the installed Robot Framework, use this code
6. You might face a scenario where you would need to use a specific version of the Robot Framework, you can accomplish this by mentioning the specific version you need as shown below.
pip install robotframework==4.0.0 //mention the specific version
7. In time, you might want to upgrade the old version of the Robot Framework. You can use the below code to do it.
pip install --upgrade robotframework
8. In addition to verifying if the Robot Framework has been installed correctly, you might also have to check if the python libraries.
(i.)
This command will show all the python libraries installed with the robot framework version.
(ii.)
This command will show all the python libraries installed with their version numbers.
For writing automation test case:
In the Robot Framework Tutorial, we would be exploring how to perform web browser testing using Selenium Library. But for that to happen, we have to import the Selenium library file to this Robot project
Step 1: Enter the Command-Line to install Selenium
Step 2: To upgrade the pip installation with the Selenium library, enter the below-mentioned command
pip install --upgrade robotframework-seleniumlibrary
Step 3: Finally, you can check if the pip Robot framework Selenium library has been installed properly by using this command.
pip check robotframework-seleniumlibrary
IDE for the Robot framework:
We can conveniently use the following IDE’s to write the Robot Framework automation testing codes.
1. PyCharm
2. Eclipse with RED plugin
3. Visual Studio with one of the plugin
4. RIDE
In this Robot Framework Tutorial, we have used PyCharm with Selenium to write a website automation testing code.
PyCharm IDE Installation and a New Project Creation:-
Step 1: Go to https://www.jetbrains.com/pycharm/download/#section=windows
Step 2: Download the free Open-Source Community Version of the PyCharm .exe file.
Step 3: Once you run the .exe file of the PyCharm setup, you’ll see the ‘Install Location’ window. If you want to change the location from the default option, you can click on the “BROWSE” button, and select your own file setup location and click on the ‘Next’ button.
Step 4: In the 4th Window, select the 64-bit launcher and click the ‘NEXT’ button to move to the next window where you can directly click on the ‘INSTALL’ button to install all the files.
Step 5: Once the setup and jar files are properly installed and updated, tick the checkbox and finish by clicking on the “OK” button in the “Import Pycharm Settings” pop-up.
Step 6: Now, you will see the ‘Welcome to PyCharm’ window. Click on the “Create New Project” option and it will automatically generate a project fill location to set up the project. If you would like to change the location, then you may click on the file option to customize it as you please. Enter the “Project Name”, and click on the “CREATE” button.
Step 7: Finally the PyCharm Window will display with New Project.
In this Pycharm IDE, we are going to create a Robot framework for a website’s automation testing. So let’s explore how to generate a report and all the other essential actions in the upcoming topics of this Robot Framework Tutorial.
How to Create a New Robot Project and the required Setups:
In the Robot framework, we have to set up interpreters and add some plugins as well. Let see how to do those things now.
Step 1: You’ve already created the Initial Project for which you need to add the interpreters to work with the Selenium and Robot framework. So follow the below steps, and use the image as a reference if needed.
Go to File > Settings > Click on Project: “current project name” > Python Interpreter > + icon
Step 2: You need to install the following packages for this project
i. Selenium
ii. robotframework
iii. robotframework-seleniumlibrary
You have to enter these three keywords one by one in the search input field and search for the packages. Once you find them, you have to click on the “Install Package” button and wait for a while for the packages to install in the back end. Once it is successfully installed, the highlighted message will appear and you can close the window by clicking on the “OK” button in the settings window.
Note: Make sure to close the window only after installing the three packages one by one.
Step 3: Now, you have to add the “IntelliBot @SeleniumLibrary Patched” Plugin as it’ll help us find the Robot framework files and give auto suggestions for the keywords. So without this plugin, PyCharm will not be able to find the “.robot” files. The steps are as follows,
Settings > Plugin > Marketplace >Search “IntelliBot @SeleniumLibrary Patched” > Install
Once the Plugin has been installed, you have to restart the IDE for the initial configuration.
Step 4: Normally, the Project Structure contains multiple directories and each directory contains a similar kind of file. So in this Test project, we are going to create a test case inside the directory first.
Right-click Test Project > New > Directory > Enter the Directory Name
That directory will be created under the Test Project and so our Robot test cases can be created inside it.
Step 5: Create the “.robot” file under the created directory.
Right Click on Created Directory > New > File > Enter a File name that ends with “.robot”
Once the robot project has been created, the structure of the project will be how it’s shown in the following image.
Prerequisites to run Automation Test Cases in the Robot Framework:
Whenever you write automation test cases in the Robot framework, 4 sections should be there. They are
- Settings
- Variables
- Test Cases
- Keywords
Note:
- These 4 sections are must-know aspects from a beginner’s point of view.
- Each and every section starts with ‘***’ and is followed by the Name of the Section. It also ends with ‘***’.
- One (or) Two Tab spaces are needed in between the Keyword and the Test data (or) Data (or) input data because indentation is very important in Python.
- When you’re about to write a test case, you should first write the test case name and then go to the next line to write the keyword after leaving a tab space.
Step 1: Settings
You are going to import the Selenium Library in this Test case.
The first line indicates the code block and the second line uses the “Library” keyword to import the “SeleniumLibrary”.
Syntax:
***Settings***
Keyword LibraryName
Step 2: Write the Test Cases
Now, you have to write exact test cases as per your requirements. First, you have to assign a test case name, and then enter the test case information in the next line. You can write these test cases using free defined Selenium Keywords. Being a leading QA Company, we wanted our Robot Framework Tutorial to be a comprehensive guide even for beginners. That is why we’re going through everything from scratch.
Syntax:
***Test Cases***
TestCase_Name
Keyword Locators(or)data Testdata
In this Robot Framework Tutorial, we are going to perform the following test cases:-
1. Visit the Codoid Website.
2. Navigate to another page and verify it.
3. Pass the Input data.
The first step towards automation is launching your browser. Based on the browser you are using (Chrome, Firefox, etc.), you have to download the various drivers. Once you have downloaded the drivers, you have to extract the downloaded file and copy the driver.exe file path. We will be using Chrome for our demonstration.
Syntax for the Chrome launch:
***Test Cases***
TestCase_Name
Keyword driver_name executable_path= “driver_path”
There is also another method that can help us save some valuable time by reducing the time to launch the browser. You can,
1. Copy or Cut “various_driver.exe” file from your local.
2. Paste them under the Python/Scripts page under your local system as shown below.
After this step, we can directly launch the browser without pointing to the driver location in the framework. We have to write the following syntax instead of using the “Create webdriver” keyword to set up the path.
Direct browser launching syntax:
***Test Cases***
TestCase_Name
Keyword input_data (or) URL driver_name
Based on the requirements, we have to write the test steps under the test case name with the Selenium Library Keywords and respected Locators along with the test data.
In this example, will be performing the test case with limited Selenium Keywords in the robot framework. For more Selenium Library keywords please visit their website.
Step 3: Define the Variable
The variables are elements used to store values which can be referred to by other elements.
Syntax:
***Variables ***
${URL} url
@{UserCredentials} inputs
Defined variables can be replaced in test cases and we can even reuse them anywhere in the test case when needed.
Apart from this, we have a lot of variables available in Robot Framework. We have mentioned only the required ones here
Step 4: Define the KEYWORDS
Since the keywords are reusable in the Robot framework, it helps manage the changes in a faster and efficient manner. You can even create our own keywords here.
Syntax:
***Keywords***
Keyword_Name
Selenium_keyword locators input_data
Once you write your own keyword, you can replace the test case steps with the keywords.
Even once all of this is done, we won’t be able to run our project directly in the PyCharm Robot framework. We would have to go to the Terminal and use the below command
robot directory_name\file_name.robot
Once the project has been successfully executed, the output.html, the log.html, and the report.html files along with the screenshots for each test case will be automatically created in our project structure. So we can copy and paste it into any browser to check the report of the execution of our project.
Here we have executed all the test cases. But if you are looking to run a separate test case, you can do so by using Tags. You should include the tags with the name of your choice inside the test cases section under the test case name as shown in the below image.
You can either run one or a particular set of test cases by using this command in the Terminal.
For one particular Tag:
robot --include=Tag_name directory_name\file_name.robot
For Two Tags at a Time:
robot –i Tag_name -i Tag_name directory_name\file_name.robot
So using tags you will be able to get more concise reports as you can execute the test cases based on priorities. Refer to the following images to get a better understanding of what we mean.
Conclusion:
We hope you have found our end-to-end Robot framework tutorial to be useful as the Robot framework is an easy-to-understand open-source tool that provides a modular interface to build custom automation test cases based on your varying requirements. So it has been widely adopted by various large organizations for this very purpose, and for good reason.
