Blog

Mobile App Testing Checklist

In general, using checklists is a great way to keep all your content organized. Likewise, mobile app testers can make use of this checklist to keep track of the level of progress and ensure that all the important tests are performed without fail. It can also be very useful in preventing testers from repeating tests that they have already done.

• App Interruption
• Testing under different Network Conditions
• Check Battery Usage & Resource Utilisation
• Usability
• Content
• Gestures
• Permission
• Performance
• Accessibility for Mobile Apps

App Interruption

Smartphones of today are designed to help their users multitask with ease. In a fast-paced world, the end-user will not be happy with your application if it doesn’t handle the various interruptions that it might face during everyday usage. So you can add the below-listed app interruption test scenarios to your test suite to avoid bad reviews & eventual app uninstallation.

1. Switch to a different app and come back again to your app screen to see how it reacts.

2. Answer or reject an incoming call when you are using your app to ensure that the app doesn’t crash or restart.

3. Click on another app’s push notification and cause an interruption while you are using your app to verify if the app returns to the same state and screen when you left it.

4. If your mobile app requires to be connected to a Bluetooth device or any other hardware, then remove the device while it is being used and see how your app handles the interruption.

5. Abruptly close the app when you are in the middle of a purchase transaction and reopen it. The app should show whether the transaction was a success or a failure.

6. Set an alarm to ring while your app is being used to ensure the app’s screen does not hang or freeze.

7. Play a video or bring your app to the foreground and leave the device unattended until it auto locks or goes to sleep. Unlock the device and check the app is where you have left.

8. Use your app when the battery is nearing the low battery threshold. You will be able to ensure that the mobile app does not crash or freeze once you have taken action on the low battery alert.

9. Test how your app reacts while downloading a large file in the background.

10. Update the app to the new version by clicking on the update available alert to see if it crashes.

11. Your app shouldn’t crash or freeze when it tries to access a corrupted file from the storage.

Testing under different Network Conditions

Did you know that 20% of mobile apps crash due to network issues? The root cause of such an issue is testing the mobile app under optimal conditions and not considering the real-world conditions. In order to ensure your mobile app’s stability, you need to test how your app adapts to the different network conditions.

1. Switch to Airplane mode while uploading a file or when your app screen is loading.

2. Change the network from 4G to 3G and vice versa to check how your app handles the change in the network.

3. Change the network when a video is being played.

4. Switch from WiFi to Mobile Data and vice versa to see how the app is impacted.

5. Change the Data connection between SIMs(SIM 1 to SIM2 and vice versa) while using your app.

6. Turn off your Data connection or WiFi while using your app to ensure it doesn’t crash or freeze as the offline functionalities shouldn’t be affected.

7. Check your app behavior when it enters or exits a Dead zone. Dead zones are usually areas where mobile phone services are not available. (Scenario for Real device testing)

8. Your app might not face any issues while downloading large files when the data connection is strong. But it should also be able to download even when the network isn’t that strong.

Check Battery Usage & Resource Utilisation

No end-user will like an app that drains their battery or uses too much of their device’s resources. So ensuring that your mobile app performs efficiently with limited resources is important. But testing such parameters using emulators and simulators is not possible. Being a top mobile app testing company, we overcome this challenge by using real devices for mobile app testing. We will be exploring the various tools that will be coming in handy for this purpose. But first, let’s take a look at the different test scenarios that will help you identify battery drain and resource management issues.

1. Test the app with low network bandwidth as it will be helpful in inspecting if there is any rapid battery drain.

2. Open and keep your competitor apps active in the background while you open or use other apps to see how the phone shares its resource between all the apps. For example, Online traders are known to use multiple trading apps simultaneously. So if your mobile app is for trading, it should be able to withstand extreme CPU load conditions when other competitor trading apps are active and open.

3. Go to any screen which supports infinite scrolling and scroll for some time to check how the app utilizes the device’s resources.

4. If you’re testing a camera app, then open the other camera apps and keep them running in the background to switch between those apps when the camera is active.

5. Check how much battery, RAM, and data your app (uptime usage) consumes when used as a typical end-user for both the foreground and background scenarios.

6. Try to download files using your mobile app when your phone does not have enough memory to accommodate them.

7. Compare battery usage after removing the cached files.

8. Monitor battery usage when your app calculates the device’s location.

9. Ensure that your app responds to the device’s battery saver mode in a way that avoids using the limited power unnecessarily.

10. If your app prefers to do power-intensive tasks once the phone battery is sufficiently charged, check whether those tasks are invoked when the battery is fully or sufficiently charged.

11. If a mobile app reads data from a sensor even when the app no longer needs the sensor data or when it is in the background, the phone battery will be consumed unnecessarily. So ensure your app does not use sensors when they are not in use.

12. Ensure that any third-party libraries used in your app don’t waste resources or perform any actions more often than needed.

Usability

Sometimes, a mobile app might have all the features an end-user might need. But if the usability of all those very useful features is low, the end-users will have a bad user experience. So let’s explore the various test scenarios that will be instrumental in ensuring that the best user experience is delivered without any exceptions.

1. Autofill is such a useful feature that will positively impact the user experience. So ensure that the users are able to input any forms quickly using the autofill suggestions.

2. Check if the buttons are large enough to click.

3. Provide enough spaces between the menus and buttons to avoid accidental clicks.

4. Test if all the buttons or clickable elements are responsive as the users will be annoyed if nothing happens after clicking a button.

5. Ensure your app does not have deep navigation.

6. If a user has provided any wrong inputs, ensure your app highlights the errors and performs a horizontal shake to alert the user.

7. Ensure the search term is still displayed even after showing the results.

8. The order and position of the buttons and text boxes ensure a seamless flow for the end-user.

9. Check if the menu icons are displayed with labels.

10. Since most users operate their smartphone using just their thumb finger, test your mobile app using only one finger to see how convenient it is.

Content

1. If your app has a news feed or has similar social media functionalities, ensure that the latest content is shown in the feeds and that appropriate alerts are sent to the user when the app remains unused for a while.

2. Make sure to not overstuff the content in both your mobile app screen and in your app’s notifications as readability is very important.

3. Ensure that the font style, color schemes, and styles are uniform across the entire mobile app.

Gestures

1. Ensure that the gestures are easy to reproduce as they shouldn’t confuse your users.

2. Most mobile users are familiar with known gestures like Tap, Double Tap, Swipe, Slide, Drag, Flick, Long press, Pinch, & Rotate. So prefer commonly known and used gestures are instead of custom gestures.

3. Check if your app works smoothly when you press multiple touch points simultaneously.

4. Now that gestures like side screen swipes and bottom screen swipes have been introduced to replace the back, home, and recent apps buttons in Android devices, it is important that your app’s navigation or features are not impacted by it.

For example, if your app has a crop feature, then keeping the crop lines at the corners can trigger the side screen swipe and close the app instead of cropping the image.

Permission

With such a majority of the world’s population using smartphones, the need for privacy and awareness surrounding it has been on the rise. The end-users are ever more concerned about their privacy and data. So as a rule of thumb, make sure you don’t request device permissions that you do not need for your app. Apart from the privacy concerns, the dependencies that are created can also be avoided.

1. If your mobile app requires location data to function, grant the location permission and close the app. Revoke the location permission and relaunch the app to see how it functions.

2. Likewise, revoke the contact access permission and relaunch the app to see how it responds.

3. Clear the app data and check how your mobile app to see its impact.

Performance

Nobody likes to use a slow mobile app that can’t keep pace with the fast-moving world. So any mobile app that doesn’t go through performance testing will most probably get lost in the competition. Here’s the checklist of test cases to follow.

1. If your mobile app uses a contact list, import a huge list and check how your app handles it.

2. Ensure your mobile app does not crash when launched multiple times in a row.

3. Load the server with simulated traffic and perform functional testing to observe how the app works with peak load.

4. Compare screen loading times with an older device.

Accessibility for Mobile Apps

With the exception of a few, almost every adult uses a smartphone. A report from the CDC states that 1 in every 4 adult Americans suffers from some form of disability. Smartphones and mobile apps have become a basic necessity in today’s technological world as it is an integral part of our day-to-day life. So let’s explore the various accessibility-based test scenarios for both Android and iOS mobile apps.

Common to both Android and iOS

1. Ensure the buttons are keyboard operable and that they show keyboard focus visibility.

2. Validate if the buttons convey their accessible name, role, value, and state to TalkBack users.

3. Check if the links are meaningful and have a purpose.

4. Ensure the progress bars conveys their accessible name, role, and current value to TalkBack users.

5. The appearance of progress spinners must be conveyed to TalkBack users either via focus management or screen reader announcements

6. Verify if the SeekBar sliders have TextView labels connected using the android:labelFor.

7. Check whether the Switch controls have the ‘android:text’ labels spoken to TalkBack users.

8. Test to see if the dynamic content changes are notified to Talkback users through spoken accessibility announcements or focus management.

9. Ensure that your user login session does not timeout without providing any accessible timeout warning or a session extension method.

10. Verify whether alerts are conveyed through TalkBack or grab users attention

11. Validate if the Modal dialogs capture the focus of the screen reader and trap the keyboard.

12. Ensure the Accordion components announce their accessible names of both the expanded and collapsed states.

13. Check if the autocomplete suggestions are announced to the users when they appear. Also, validate if they are operable for TalkBack and Keyboard-only users

14. Make sure there is a way to stop the Carousel’s movement.

15. Ensure the current rating value has an accessible name and that the value is conveyed to the end-user as well.

16. Verify if the sorted state of a sortable table column headers is conveyed to the users.

17. Ensure the state of the toggle button is conveyed to the end-user.

18. The label text should be visible to the end-user at all times.

19. Check whether the groups of related inputs convey their group legend text to TalkBack users.

20. Ensure that the native date picker controls are used over plain text inputs or custom date picker controls.

21. Validate if the radio buttons convey their individual label, group legend, and checked state to the end-users.

22. Check if the Select dropdown controls have an associated visible text label.

23. Make sure the form validation error messages are conveyed to the user when the errors are displayed or when a user is focused on invalid inputs.

24. Check if the data tables are in logical reading order and if the data cells are conveying their row and column header text to TalkBack users.

25. Videos should include closed captions for the deaf and ‘hard of hearing’ users.

26. Likewise, the videos must also have audio descriptions or separate audio-described versions for users who have vision disabilities.

27. If you have used any decorative or redundant images, make sure that such images don’t have any alternative text equivalents as they should be hidden from the screen readers.

Exclusive to iOS

1. Check if the click or touch target size is large enough without having to zoom in.

2. Make sure that the touch targets don’t overlap.

3. Even if there are any compact touch targets, ensure they are not placed adjacent to each other.

4. If the active elements have the same target, make sure those elements are grouped into a one-touch target.

5. Ensure that the application doesn’t rely on kinetic motion (detected by the accelerometer) alone.

6. Even if there are any motion-sensitive features, verify if they can be disabled by the users.

7. Check if the application is fully or heavily reliant on voice control alone.

8. Likewise, the mobile app shouldn’t be too reliant on gestures alone too.

9. Check if the movement of the physical device causes any non-essential visual motion or animations like parallax effects.

10. Validate if the users are able to disable any non-essential motion-sensitive visual effects.

11. Make sure that the content doesn’t flash more than thrice every second. It can be allowed if the flashing content is very small or if the contrast of the flashes doesn’t violate the general flash thresholds.

12. Verify if the alternative text is available for custom emojis in custom keyboards.

Mobile App Testing Tools

There are various open-source and paid tools when it comes to mobile app testing. You might come up with some of the mobile app testing strategies, but successful implementation of them depends on the mobile app testing tools that are available and your proficiency in using them. That is why we have a dedicated R&D team that will identify the various new tools and analyze them to see if they can enhance our testing process. Since we have used various mobile app testing tools in our very own projects, we are aware of some of the best tools used by numerous mobile app testing companies all over the world. So let’s take a look at the list of the best mobile app testing tools, starting with Appium.

• Appium
• TestComplete
• Robotium
• Xamarin.UITest
• Calabash
• Ranorex Studio
• SeeTest
• EggPlant
• Apptim
• Perfecto
• Frank
• Espresso

Appium

Appium is one of the most popular open-source automation tools used by testers. Like how we have Selenium for performing automation tests on Web pages, Appium is instrumental in helping us execute automation tests on different mobile devices. Since most organizations are keen on developing both web-based and mobile-based apps, using Appium for mobile app automation is one of the best practices. So let’s explore the many useful features and benefits of Appium that make it so popular.

Features

• Appium is an open-source tool that has great community support.
• It can be used to perform tests on both Android and iOS apps.
• You can use Appium to test hybrid, native, and web-based applications.
• Appium’s cross-platform testing capabilities make Android testing on macOS, Windows testing on Linux, and iOS testing on macOS possible.
• It supports all the frameworks and all the prominent programming languages as well.
• The Appium server processes requests from the Appium client and forwards them to the real devices where the test scripts are automated.
• It generally uses JavaScript as the server language to fetch and transfer the data by communicating with the client and the user.
• There is no need to install any application on the mobile to trigger the testing.

TestComplete

TestComplete is an Automation Testing tool used for testing web apps, native mobile Android & iOS apps, desktop apps, and API services as well. TestComplete supports various frameworks such as Keyword Driven, Data-Driven, and also BDD. It also allows the users to use various programming languages such as JavaScript, Python, VBScript, etc… for scripting. But what sets it apart is that it has a record and playback feature that enables its users to perform automation without having any scripting knowledge. TestComplete uses the GUI Object Recognition Algorithm with AI capabilities to identify the elements across the application and capture them. And If there is any change in the element’s value, it will automatically detect the exact change in the object and list it out under the Test Log.

Numerous mobile app testing companies use TestComplete in their various projects to get in-depth reports that provide the status of all the tests across web, mobile, and desktop apps. It even provides us the entire recording of the test execution that aids in the visual debugging of issues, captures the screenshot of every action performed on the application, and highlights the element on which the action was performed.

Features

• The Record and Replay feature isn’t limited to just basic actions as it can be used to even automate complex scenarios without us having to write a single line of code.
• TestComplete is also well-known for its Keyword Driven Testing which enables testers to use drag and drop gestures to write easy-to-read and reusable automation scripts.
• In addition to enabling scriptless automation, it helps a wide range of software testers to develop automation scripts as it supports numerous programming languages.
• Its Hybrid Engine helps widen the test coverage by using an AI-powered visual recognition to detect all the UI elements and execute a variety of actions.
• The Data-Driven test feature that enables retrieving data from Excel worksheets and CSV files also helps in expanding the test coverage.
• If the UI element cannot be identified using the traditional object identification approaches, TestComplete switches to an AI-powered object recognition mode. It identifies the element and manipulates user actions such as clicks, hovers, and touches to widen test coverage across platforms using Optical Character Recognition (OCR). Such features make it the ideal choice for testing charts, mainframes, PDFs, SAP, and other such technologies.
• It makes it easy to automate HTML5 elements that include Shadow DOM and Custom Elements.
• The in-depth test report logs can be exported into different formats such as HTML, JUNIT, or MHT based on the stakeholder’s preference.

Robotium

Robotium is also another great open-source test framework that can be effective when it comes to writing automated gray box testing cases for Android applications. Robotium is very much similar to Selenium, but it is built for Android using Java and the JUnit test structure. You can use Robotium to write function, system, and acceptance test scenarios for multiple Android activities. What makes Robotium a great choice for testing Android apps is that it supports various android features such as activities, toast, menus, and even context menus. You can use Robotium to test mobile apps where only the source code is available and also to test mobile apps where only the APK file is available. Robotium even provides you with the option of testing on real Android devices or on an emulator. As a mobile app testing company, we always recommend using real devices for all testing. But even if you are using an emulator, you can still make use of Robotium.

Benefits of Robotium

• Robotium makes it easy to write solid test cases with minimal time because of the shorter code.
• It also enables us to develop powerful test cases even if in situations where we know very little about the application under test.
• The readability of test cases in Robotium is greatly improved when compared to the standard instrumentation tests.
• It provides automatic timing and delays.
• Robotium automatically follows the current activity, finds the views, etc…
• Robotium can take very useful autonomous decisions like deciding when to scroll and so on.
• No additional modifications are required to the Android platform.
• Robotium exhibits fast test execution.
• The run-time binding to GUI components enables their test cases to be more robust.
• It also integrates smoothly with Maven or Ant.

Xamarin.UITest

Xamarin.UITest is another great C# based automation framework for performing UI Acceptance testing on Android and iOS applications using the Nunit framework. Xamarin.UITest achieves this by using a single shared codebase (i.e) The complete body of the source code that is needed by the given program or application to run is stored in a single place).

Features

• It is an open-source cross-platform testing framework.
• Backdoors can be used to ensure that all the texts in a given fixture have the same data.
• UI tests can be categorized into tablet-specific and mobile-specific tests for the same application.
• It has a predefined 15 seconds window for local tests and a 1-minute window for App Center tests before it throws a TimeoutException. You can even customize the waiting period based on your requirements.
• The Xamarin.UITest Automation Library allows testers to interact with the application UI and perform end-user actions such as entering text in input fields, tapping buttons, and gestures.
• The REPL (Read-Eval-Print-Loop) tool can be used to dynamically test expressions to evaluate, execute and log the results. The expressions can even be copy-pasted into your UI testing projects.
• You can ensure that your tests don’t fail on App Center tests by employing the embedded files and included files methods that help include the files with your test upload.
• Hybrid mobile apps that use Web Views to display the HTML are harder to test as you’ll need to find a reference to the Web View first and then to the DOM elements it contains. However Xamarin.UITest has APIs that can interact with Web Views in such a situation.

Calabash

Calabash is an open-source automation testing framework that is developed and maintained by the Xamarin team. Being developed by the same team it has similar features as Xamarin.UITest such as cross-platform testing of both native and hybrid apps. Calabash is based on Behavior Driven Development (BDD) which describes the application’s behavior. So Calabash tests are executed on real mobile devices to get accurate results. Feature definition files and Steps definition files are instrumental in achieving mobile automation in the Calabash framework. Calabash’s tests are written in Gherkin, backed by Ruby language code, and run in the context of the Cucumber Framework. It supports about 80 different natural language commands (controllers), and new controllers that can be implemented in Ruby or Java language.

Features

• It is an open-source framework that can be used to test both native & hybrid Android and iOS apps.
• Since it supports Cucumber, you can write tests in a natural language that can be easily understood by everyone on the team including business experts and non-technical QA. Additionally, it also creates a bridge that allows Cucumber tests to be run and validated on Android and iOS.
• It can interact with apps like Espresso or XCTest.
• Integration with continuous integration or continuous delivery tools such as Jenkins is easy.
• It can work with any Ruby-based test framework.
• Calabash provides real-time feedback and validation across many different factors like OS versions, hardware specifications, OEM customizations, chipsets, amount of memory, and real-environment conditions such as backend integrations and network conditions.

Ranorex Studio

Ranorex Studio is a GUI test automation framework that can be used to test mobile, desktop, and web-based applications. It uses standard programming languages such as VB.NET and C#. What sets Ranorex Studio from the rest is that it includes the Ranorex recorder, object repository, Ranorex spy, code editor, and debugger in a single environment. It also provides great XML-based UI test execution reports that include screenshots of failed test cases. In comparison, Ranorex is better at Recording, Replaying, and Editing User Performed Actions.

Features

• Ranorex Studio is an open-source tool developed by Ranorex GmbH.
• It supports technologies like Silverlight, .NET, Window forms, Java, SAP, WPF, HTML5, Flash, Flex, Windows Apps (Native/Hybrid), and iOS, Android
• It runs on Microsoft Windows and Windows Server.
• Xpath technology is used for the identification of objects.
• Object-Based Capture or Replay Editor offers a simple procedure to create automated test scripts that enable even non-coders to create tests without any hurdles.
• It can test and validate web applications across popular browsers like Chrome, Safari, Firefox, and Microsoft Edge
• It can be used for regression testing in continuous build environments to find new software bugs much faster.
• The reports generated by Ranorex can be used to easily reproduce bugs and maintain the tests as well.
• It provides great support for older OS versions. (Supported on Android 2.2 & higher, and on iOS 5.1 & higher.)

SeeTest

SeeTest Automation is a fantastic mobile app testing automation tool that supports both Image-Based and Object-Based recognition. It begins by connecting a real mobile device and a corresponding emulator or a cloud device. Once the device is connected, the control buttons will appear at the bottom of the computer screen that allows you to record and edit scripts while performing a test scenario. It has its own reporting mechanism that contains screenshots and video recordings of test executions. It also helps to evaluate the execution reports and export the selected test code to any of the testing frameworks.

SeeTest has the commands for almost all the actions which are performed on a mobile device. It includes actions such as changing the device settings, interacting with a non-instrumented application (SMS, Contacts, Dialler), launching & killing the application life-cycle, and so on. These actions are enabled by SeeTest controlling the device’s springboard.

Features

• Supports automation of iOS, Android, Windows Phone, and BlackBerry applications.
• Provides client libraries for the languages such as Java, C#, Perl, Python in order to develop automation scripts.
• The mobile device can be connected either by using USB, Wi-Fi, or the SeeTest cloud.
• It supports testing frameworks like UFT, TestComplete, RFT, C#, JUnit, Python, and Perl.
• It can test Android wearable devices with non-instrumented mode.
• SeeTest can quickly and easily validate layouts on a mobile device.
• The password encrypt feature allows you to create tests that access a real account without revealing the account’s credentials.

EggPlant

Eggplant is a GUI test automation tool that employs an image-based approach to GUI testing to test the application with better and faster execution. It can be used to validate functionality based on the user’s perspective and check the performance of the mobile app on any device or operating system. This software uses a two-system model where one is the Eggplant testing tool that is installed and runs on a Desktop, and the second is the System under test (Mobile device). It uses a virtual network computing connection to bridge the above two systems. Eggplant’s standout performance provides strong test composition, environment management, dynamic control, and result in analytics to JMeter’s existing scripting capability.

Features

• Supports Mac, Windows, and Linux
• It can be used for Mobile Testing, Cross-Platform Testing, Internet Application Testing, and Performance Testing
• It can interact with any device of your choices such as a mobile, tablet, desktop, server, and Internet of Things devices
• Integration with Application Lifecycle Management (ALM) software like HPE Quality Center, IBM Rational Quality Manager, Bamboo, IBM UrbanCode, and Jenkins is a big plus.
• Eggplant version 11 was integrated with the OCR (Optical Character Recognition) engine and even introduced macOS X Lion support.
• Version 12 introduced the functional user interface redesign, consolidating the suite interface and scaling search which allows for testing across different sizes of screens with the same image.
• Eggplant version 14 has database integration via ODBC (Open Database Connectivity).
• Version 15 started supporting Tables for keyword-driven testing and Turbo Capture for script recording. It is worth noting that the VNC server in version 15 for Android allows its users to test an Android Smartwatch.

Apptim

Apptim is a great open-source tool that can be used to test your mobile app’s performance. It can be used to automatically measure app render times, power consumption, and resource utilization while capturing vital data such as crashes and so on. It is a desktop application that can be used to test native Android apps on Windows and native iOS & Android apps on macOS. In addition to supporting both native Android & iOS mobile apps, it also provides its users with a web-based workspace where they can save testing reports and share them with the entire team. So let’s take a look at the standout features of Apptim.

Features

Apptim is very easy to set up all thanks to its automated installation and low storage space requirements.

It also becomes a viable tool on your phone for keeping all features operational at peak performance.

• Apptim has a user-friendly UI that is suitable for experts and novice users.
• It establishes the essenial thresholds for the mobile performance KPIs.
• Apptim has great security features as it automatically detects malware or spyware, quarantines them, and deletes them.
• It analyses and evaluates performance tests.
• Apptim provides a detailed report on functionality along with repair options.
• Reports on comparison functionality can also be obtained if needed.
• It provides hassle-free mobile app usage that minimizes time wastage.
• It effectively replaces standalone emulators by running automated tests that are remotely powered.
• Based on the feature, it even has the ability to employ both online and offline configurations.
• It provides unlimited OS usage and timely facility updates as well.

Perfecto

Perfecto is one of the best cloud-based testing tools that has the ability to create a constantly updating test environment that keeps up with the frequent mobile and browser releases. It does so by allowing its users to use the most latest Android & iOS platforms instantaneously. It even works great with the newest versions of popular browsers such as Chrome, Firefox, and Safari. Perfecto’s test analysis report makes it easy to identify the source of a test failure by employing root cause analysis. Now let’s take a look at the set of its standout features.

Features

• Perfecto supports mock location for iOS.
• It enables parallel test execution that can save loads of time.
• Perfecto provides access to mobile settings.
• Perfecto allows you to install an unlimited number of mobile apps.
• It supports JIRA integration for reporting and managing all the bugs.
• It can be used to automate native, web, and hybrid mobile applications.

Frank

Frank is an open-source library that can be used to perform functional tests for iOS mobile apps. Frank embeds an HTTP server into the test version of the mobile application that runs once the app is started. Through this internal HTTP server, the cucumber part of Frank will send commands to the app to remotely perform actions such as button taps, filling texts, or looking for UI elements that you want to see on specific views. It can be used to write structured text tests, acceptance tests, and requirements with the help of Cucumber.

Features

• Frank includes a powerful “app inspector” known as Symbiote to get detailed information on the running app.
• Getting the iOS app set up for Frank does not take more than 10 minutes.
• It can also be used to write tests for Mac apps and the process of writing those tests is similar to writing tests for iOS apps.
• It has pre-defined steps that can be used right away by the user.
• It allows you to record the Video of your cucumber run.
• You can run your tests on both Simulators and Real (Physical) Devices.
• Its continuous integration support enables you to run your tests on every check-in.

Espresso

Espresso is an Android exclusive testing framework that has been developed by Google, the company that makes Android. It provides a simple and flexible API to automate the UI in an Android mobile app. Espresso tests can be written in both Java and Kotlin languages. It even supports both Android native view and web view. Though it is intended to be used by developers for testing the UI of Android mobile apps, its features make it one of the best mobile app testing tools as well.

Features

• The tests are highly scalable and flexible.
• Espresso is integrated with Android Studio Applications.
• Test scripts are far more stable and faster than Appium when it comes to UI testing.
• UI test scripts are customizable and easy to maintain as well.
• It has the Espresso Test Recorder that can record our manual user interactions in the UI to create automated tests without any coding.
• Espresso also interacts & automates with web elements inside the WebView components.

Mobile App Automation Testing

We are a mobile app testing company that understands the value that automation brings to the table. But we also know that automation is no easy task and that it requires a lot of effort to implement it successfully. Appium is one of the best automation tools that every tester must know when it comes to mobile app automation testing. So we will be exploring how to set up Appium and learn how to automate various gestures as well.

• Appium Setup for IOS Device
• How to Automate Gestures

Appium Setup for IOS Device

Required Software

• Appium version: 1.22.0
• MAC OS/version used to run Appium: 11.6
• Node.js version: v15.14.0
• Xcode version: 13

Once the required tools & software are installed, enter the below list of command lines one by one in the terminal,

      $ npm install -g appium
      $ npm install -g appium-doctor
      $ brew install libimobiledevice --HEAD
      $ brew install ideviceinstaller
      $ npm install -g ios-deploy
      $ gem install xcpretty
      $ cd /Applications/Appium Server GUI.app/Contents/Resources/app/nodemodules/appium/nodemodules/appium-webdriveragent
      $ brew install carthage
      $ npm i -g webpack
      $ mkdir -p Resources/WebDriverAgent.bundle
      $ ./Scripts/bootstrap.sh -d
      

Once you have executed the above commands without any errors in the terminal, you’d have to install the ‘WebdriverAgent’ iOS app on a real device (iPhone).

• Connect the iPhone to your MAC Machine.
• Open the ‘WebdriverAgent’ folder using the Finder.



• Open the ‘WebDriverAgent.xcodeproj’ file from that folder to open the XCode on the webdriveragent project.



• Select the connected device in the ‘WebdriverAgentLib’ which is next to the build icon.



• Select the WebdriverAgent Project and select the basic tab, choose the ‘iOS Deployment Target’ version as the same version of your device or the lower version of it.



• Create an Apple ID and add it to the XCode(XCode -> Preference -> Account Tab)



• Select the Target as ‘WebdriverAgentLib’ and keep the bundle identifier as unique. Select the Team as ‘Apple ID’(Added in the Preference’s Account) and the Deployment target as the connected iPhone OS version.



• Select the Target as ‘WebDriverAgentRunner’ and keep the product bundle identifier as unique in the Build Settings tab and the Deployment target as the connected iPhone OS version.



• Select the General Tab in the ‘WebDriverAgentRunner’ Targets and select the Team as ‘Apple ID’(Added in the Preference’s Account). The Signing Certificate should be displayed as ‘iPhone Developer’.



• Follow the above two steps for the rest of the listed targets.
• Build the project by pressing the ‘Play’() button and the ‘Build Succeeded’ message will appear(Avoid the warnings).
• Get the UDID of the connected Mobile.
• Enter the given below command with your phone UDID in the terminal which has already opened the WebDriverAgent folder.

      $ xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination 'id=
      <YOUR_UDID>
      ’ test -allowProvisioningUpdates
      

• You need to trust the developer in the device of the WebdriverAgent App (Settings → General → Device Management → Developer App → Trust Developer)
• Run the above ‘xcodebuild’ code in the terminal. Once the below image is displayed in your terminal, you can stop the process by pressing ‘CTRL + C’.
• The WebDriverAgent app will be installed on your device and you will be able to continue your testing in Appium. Use the below config in Appium while testing the mobile application.

      {
      "deviceName": "iPhone",
      "platformName": "iOS",
      "VERSION": "12.0.1",
      "autoGrantPermissions": true,
      "udid": “
      <YOURS>
      ”,
      "automationName": "XCUITest",
      "bundleId": "com.ios.message",
      "noReset": true,
      "useNewWDA": false,
      "bootstrapPath": "/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent",
      "agentPath": "/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent/WebDriverAgent.xcodeproj"
      }
      

How to Automate Gestures

Since every smartphone has a touch screen, gestures are the highly important input actions needed to use a mobile app. So in this section, you will learn how to perform mobile gestures using Appium.

Tap

Tap gesture invokes or selects an item the same way a user places a finger on the item and removes it immediately. You can define an element to select or define the coordinates where the tap has to happen.

      TouchAction touchAction = new TouchAction(driver);
      touchAction.tap(tapOptions()
      .withElement(element(androidElement)))
      .perform()
      

Tap using Coordinates

      TouchAction touchAction = new TouchAction(driver);
      touchAction.tap(PointOption.point(1280, 1013)).perform();
      

Long Press

Long press is commonly used to display the context menu or to show options related to an item. So when a user places their finger on an item and holds on for a second or two, it should show the options related to the item.

      TouchAction touchAction = new TouchAction(driver);
      touchAction. longPress(LongPressOptions.longPressOptions()
      .withElement (ElementOption.element (element)))
      .perform ();
      

Swipe

Swipe is also a common gesture that will be needed to test a mobile app’s functionality. You can do so by using the below code.

      TouchAction swipe = new TouchAction(driver)
      .press(PointOption.point(972,500))
      .waitAction(waitOptions(ofMillis(800)))
      .moveTo(PointOption.point(108,500))
      .release()
      .perform();
      

Swipe using element

      int startX = startElement.getLocation().getX() + (startElement.getSize().getWidth() / 2);
      int startY = startElement.getLocation().getY() + (startElement.getSize().getHeight() / 2);
      int endX = endElement.getLocation().getX() + (endElement.getSize().getWidth() / 2);
      int endY = endElement.getLocation().getY() + (endElement.getSize().getHeight() / 2);
      new TouchAction(driver)
      .press(point(startX,startY))
      .waitAction(waitOptions(ofMillis(1000)))
      .moveTo(point(endX, endY))
      .release().perform();
      

Drag & Drop

Drag & Drop is such a convenient feature that results in great functionalities. So let’s take a look at the snippet you’ll need to automate the gesture in Appium.

      TouchAction swipe = new TouchAction(driver)
      .press(ElementOption.element(element1))
      .waitAction(waitOptions(ofSeconds(2)))
      .moveTo(ElementOption.element(element2))
      .release()
      .perform();
      

MultiTouch

Some apps don’t allow or support simultaneous touch points on the screen. However, mobile gaming apps should support multiple touchpoints for the game to be usable. In that case, you can’t go ahead without testing it. So we have listed the snippets you’ll need to test using multiple touchpoints actions such as zooming in, zooming out, and pinching.

      TouchAction touchActionOne = new TouchAction();
      touchActionOne.press(PointOption.point(100, 100));
      touchActionOne.release();
      TouchAction touchActionTwo = new TouchAction();
      touchActionTwo.press(PointOption.point(200, 200));
      touchActionTwo.release();
      MultiTouchAction action = new MultiTouchAction();
      action.add(touchActionOne);
      action.add(touchActionTwo);
      action.perform();
      

Touch Gestures to Zoom In and Out on Google Maps

      WebElement map = driver.findElement (By.id ("<>"));
      final int x=map.getLocation().getX()+map.getSize().getWidth()/2;
      final int y= map.getLocation().getY()+map.getSize().getHeight()/2;
      Zoom
      TouchAction finger1= new TouchAction(driver);
      finger1.press(PointOption.point(x,y-10)).moveTo(PointOption.point(x,y-100));
      TouchAction finger2= new TouchAction(driver);
      finger2.press(PointOption.point(x,y+10)).moveTo(PointOption.point(x,y+100));
      MultiTouchAction action= new MultiTouchAction(driver);
      action.add(finger1).add(finger2).perform();
      Pinch
      TouchAction finger3= new TouchAction(driver);
      finger3.press(PointOption.point(x,y-100)).moveTo(PointOption.point(x,y-10));
      TouchAction finger4= new TouchAction(driver);
      finger4.press(PointOption.point(x,y+100)).moveTo(PointOption.point(x,y+10));
      MultiTouchAction action2= new MultiTouchAction(driver);
      action2.add(finger3).add(finger4).perform();
      

Mobile App Performance Testing

Both the highs and the lows matter when it comes to performance testing as the mobile app must be able to handle both large amounts of load and perform the different actions despite having limited resources. The mobile app market is very unique due to the large number of devices available in different price brackets. Apart from the smartphone’s capacity, factors such as latency, bandwidth, CPU usage, and so on also matter. So testing your mobile app’s performance on a wide range of high-end and low-end devices under real-world conditions is critical. Here are a few KPIs that will help you ensure your mobile app’s performance and stability.

Mobile App Performance testing KPIs

The Key Performance Indicators that we are going to explore have been chosen based on the years of experience we’ve had in mobile app testing.

Ensure Short Load Time

Did you know that around 61% of users expect their apps to start within 4 seconds and respond within 2 seconds? So if you keep your users waiting, there are high chances for the user to just close the app and not even try your app ever again. But it is not just about testing to see if the mobile app responds within 2 seconds. You have to ensure that there is a loading animation or some kind of effect that lets the user know that the action is in progress. A simple switch without any transition might make the app seem slow to the end-user.

Ensure Quick Render Time

Render time might seem very similar to the load time that we saw earlier, but the key difference here is that it is the time taken not just for the screen to load. Rather, it is the time taken by the mobile app screen to become usable. In other words, the user must be able to interact with the mobile app screen without any issues. For example, users will find it annoying if a text field doesn’t allow them to type even though it is visible. So you can perform a stopwatch test to measure the rendering time of the entire page and verify if it is in the acceptable range.

Check for Dropped Frames

FPS (Frames Per Second) is a metric that is used to measure the visual fluidity of moving images in general. The lower the frame rate, the less fluid the motion is. So it is vital for mobile app testers to ensure that the mobile app doesn’t drop any intended frames when it is being used. FPS is not just for gaming applications as seamless transitions and animations are important for all kinds of mobile apps to provide the best user experience. The optimal frame rate is 30 frames per second for regular mobile applications and 60 frames per second for gaming applications.

Avoid Latency & Jitter Issues

It is common knowledge that data is divided into multiple packets of information when they are transmitted over any connection. So the back and forth transaction of these packets of information has to happen rapidly for the mobile app to have low latency. Data-heavy files or audio/video files are general types of data that will experience latency issues. So make sure to test for such latency issues to ensure the best user experience.
Jitters are also very similar to latency issues. The difference here is that latency measures only the time it takes for the data packet transfer. Whereas, jitters denote the level of inconsistency in latency. Jitters are frequently caused due to factors such as network congestion, route changes, and so on.

Prevent API Request Latency Issues

Apart from the general latency issues, API request latency issues are also very important as mobile apps are heavily reliant on APIs for their various functionalities. API request latency is the total amount of time it takes for an API system to respond to an API call. You can measure the value by calculating the time it takes for a request to be received by the API gateway and returned to the same client. Such latency issues will result in slow load times that could severely impact your mobile app’s success.

Prevent CPU Choking

Since the CPU (Central Processing Unit) is the unit in charge of executing all of an application’s instructions, it is a shared resource. So if your app unnecessarily chokes the CPU, the user might experience sluggishness or battery drain on the whole and not just in your app. If using a slow app annoys the user, then an app that slows your smartphone down will definitely be uninstalled instantaneously. You can keep track of your mobile app’s CPU consumption by using the below ADB command.

Syntax: adb shell dumpsys cpuinfo “Package-Name”

Example: adb shell dumpsys cpuinfo ‘com.example.myapp’

Mobile App Usability Testing

We would have already explored some mobile app usability and user experience focus points in the checklists section. We will be taking a deep dive now to understand the importance of mobile app usability testing and find out how to do it effectively as well. Assessing a mobile app’s usability in the development or programming phase is impossible as it can be achieved only through testing. So let’s find out how to do it.

• How to Perform Usability Testing?
• The User Experience Focus Points
• Benefits of Usability Testing

How to Perform Usability Testing?

Identify real users or representatives of the target audience who will be able to evaluate your product in real-world use cases. Make sure to observe how the users are using your app in different scenarios and identify the areas of improvement for your team to work on. Finally, interview the real users or representatives with questionnaires and prepare a feedback report to share with your team.

Usability Testing will help you answer critical questions such as

1. Is the mobile app useful?

2. Does the mobile app add value to the target audience?

3. Is the mobile app easy to use?

4. Does the mobile app fulfill its purpose effectively and efficiently?

The User Experience Focus Points

The objective of performing usability testing for your mobile app is to ensure the best user experience. Here are the focus points that will help us achieve this goal.

Context

A tailor-made mobile app can help enhance the user experience, and the best way to do it is by understanding the purpose of the app and its targeted user base. You can do so by answering the following questions.

1. Who will use the app? What’s special, unique, or different about them?

2. Where will the app be used? Will there be different uses or needs in different countries or rooms?

3. When will the app be used? Will it be used differently at different times of the day or year?

4. What tasks will be performed using the app? What other alternatives are there to perform the same task?

5. What are your users trying to achieve using the app?

6. Which parts or functionalities of the app are used the most? What do the people using the app want it to do?

7. How is the app used by the end-user? Is the app used in the way that it was originally intended?

For example: Let’s say you have released your mobile app after testing it using emulators on developer machines that have good network connectivity. By doing so, you are missing out on the context of the environment in which the app will be used. So your mobile app might face issues in real-world conditions. That is why you should test your app in all network conditions where the users will use the app.

Input – There should always be multiple options for a user to give inputs. If your mobile app is in need of the user’s location, it should be able to get the required input either by using the device GPS or by allowing the user to pin the location or type the street address manually.

Output – The output of your mobile app goes more than what’s displayed onscreen. For example, if your mobile game doesn’t produce the needed sound effects on time, it will definitely debase the user experience.

Responsiveness – We have already established how important responsiveness is as nobody likes to use a slow app. But since certain heavy operations will need time for it to function, we can avoid showing a blank screen, frozen screen, or a loading icon by loading the screen component by component. It is evident here that responsiveness is not just about the speed at which the page is loaded. So make sure to show why you are making the users wait when it is busy loading heavy operations.

Connectivity – Nowadays, most apps require a connection to the server to retrieve data. If connectivity is disrupted during a transaction, your app should display the appropriate error message to avoid the user from assuming that the issue is with the app. In addition to that, the app must be able to resume from where he/she left or inform what happened to the failed transaction.

Resources – Your app should not drain the battery quickly or dump your phone’s memory with unnecessary files.

Benefits of Usability Testing

1. By performing usability testing, you can understand your users and set benchmarks and usability standards for future releases.

2. Usability testing can be used to boost sales as it validates if a product is simple enough to use to make the end-user happy. It will also aid in minimizing the number of support calls from users.

3. Identifying usability issues before an app release minimizes the risk.

Compatibility Testing

According to a statistic, there were a whopping 7.1 Billion smartphone users in the year 2021. It was also reported that this number is expected to grow in the coming years. So once your mobile app is launched, it goes without saying that users will use it on different devices and platforms. So performing compatibility testing to ensure that your mobile app is compatible across various different device configurations is important. Without that, the time and effort you spent on planning and developing your app will go in vain. Before we find out how to perform compatibility testing, let’s take a closer look at the different issues the lack of compatibility testing will cause.

• Common Compatibility Issues
• How to do Mobile App Compatibility Testing
• Types of Compatibility Testing

Common Compatibility Issues:

• Content will not fit well on devices with different screen sizes.
• The look and feel of the User Interface (UI) might differ.
• Scroll bar issues.
• Frames, media content, or tables may be broken.
• Different navigation methods might be required.
• Installation and upgrade issues.
• DOCTYPE errors.
• Failure to detect outdated browsers.
• Layout issues.

To identify these issues, you’ll need to perform compatibility testing on different devices, operating systems, and browser combinations. With the number of smartphone models and variants increasing day by day, compatibility testing will play a crucial role in the success of your mobile app.

How to do Mobile App Compatibility Testing

It goes without saying that it is impossible to perform compatibility testing to cover all the existing combinations. So, start by defining priorities after thoroughly analyzing the target audience, existing users, current markets, popular gadgets, product specifics, and business objectives. Once you have the list of high-priority combinations, use the below focus points to get the best compatibility testing results.

Version Testing – Ensures the mobile app is compatible with different versions of the mobile operating systems.

Hardware Testing – Test the mobile application with various hardware configurations such as sound, screen, sensors, indicators, connections, buttons, available memory, power management features, and connected devices such as modems, disc drives, and serial ports.

Software Testing – Test if your mobile app works with other related apps for functionalities like sharing and so on.

Network Testing – Test the mobile app under different network conditions such as 3G, 4G, 5G, and Wi-Fi.

Operating System Testing – Confirms the software app performs appropriately with different operating systems such as Android OS, Blackberry OS, Apple iOS, Windows mobile OS and so on.

Device Testing – Ensure the app is compatible with different types of peripheral devices connected through USB or Bluetooth. SD Card, and others.

Types of Compatibility Testing

There are basically two types of Compatibility testing.

Backward Compatibility Testing is the process of testing the mobile app on older versions of devices and operating systems to make sure that your app can run on them as well. For example, if your mobile app works on Android 11, it should also be backward compatible with a device running on Android 10.

Forward Compatibility Testing is the process of testing the mobile app with new and upcoming versions of hardware and software. Since this is the opposite of the previous type, an app that is working fine on Android 10 should also function well if the device updates to Android 11.

Conclusion

Knowing when a mobile app is ready for release is very essential when it comes to successful app deployment. That is why we had decided to create a categorized mobile app testing checklist as it would be helpful to concentrate on real-world concerns such as app interruptions, resource utilization, usability, and so on. Since there is more to mobile app testing than following a mobile app testing checklist, we have also covered the top mobile app testing tools that will help you achieve your goals. A mobile app testing company must be able to render effective app testing services to the client irrespective of the varying needs. So the mobile app testing company should have strong automation capabilities, follow the best mobile app testing practices, and be familiar with the important tools.