Mobile Testing Types and Approaches
Shreya Bose, Technical Content Writer at BrowserStack - May 20, 2020
The future of mobile is the future of online. It is how people access online content now. — David Murphy, Founder and Editor of Mobile Marketing Daily
Murphy knows what he is talking about. Let’s look at some numbers.
- 57% of all digital media usage comes from mobile apps.
- 21% of millennials open an app 50+ times per day.
- Mobile apps are expected to generate $189 billion in revenue by 2020.
- 90% of mobile time is spent on apps.
Naturally, mobile testing is an indispensable part of all software testing cycles. Each app must run flawlessly on thousands of different devices and operating systems. To ensure that this is possible, mobile testing must be planned and executed with the utmost thoroughness and accuracy.
To that end, this article discusses a few types of mobile application testing that would best serve testers, developers, and in the long term, the users.
- Functional Testing: This mobile testing type aims to verify that every function of an app is working exactly as required. It focuses on testing user flows within the app, as well as each feature that comprises each user flow. Bear in mind that “only 4 out of 100 unhappy customers will complain directly to a company — the other 96 will churn without providing feedback.” Customers expect that each app will be flawless in its functioning. If they encounter a flaw, they will usually not think twice about uninstalling the app. There are multiple usage scenarios that must be included in functional testing, but some of the important ones would be:
- App installs and launches correctly
- Users can sign-up and log into the app easily
- Text boxes, buttons, menus and icons function accurately
- Push notifications render correctly and appear at the right intervals
- Any transactions or purchases should happen seamlessly
- Interruption Testing: This form of mobile testing checks how an application responds when faced with an unexpected interruption. Depending on the nature of the interruption, the application should pause and then return to its original state, or even react in a particular way. Obviously, the kind of interruptions will differ on the basis of the application under test, but some common interruptions that should be considered while testing are:
- Incoming or phone call when an application is running
- Incoming message or SMS when an application is running
- Low battery when an application is running
- The device plugged in or out of charging when an application is running
- Device shutting down when an application is running
- OS upgrade occurring when an application is running
- Loss and restoration of the network while an application is running. Interruption testing ensures that an app handles interruptions without failure or anomaly. When being used by real users, every app will have to operate along with other device functions. This means that every app will have to be optimized to deal with these device functions while running at all times.
- Localization Testing: This approach to mobile app testing is meant to test features that are dependent on the geographical location of an app. Since most apps seek to appeal to a global user base, they include localized features for the convenience of users. These features can vary from enabling different languages, enabling commerce in local currency to adherence with local laws and regulations. Localization testing checks these features to ensure that they are activated and functioning in the right locations. Customers always prefer apps with UI elements aligned with their culture, language, and device accessibility. They expect their experience to be adjusted to their localized needs and preferences. AppAnnie’s research confirms that fully localized apps do better in the global market. Localization testing is also one of the most challenging mobile app testing types since most QA teams lack adequate access to test coverage and resources necessary for its implementation.
- Memory Leak Testing: A memory leak refers to a situation in which the app fails to return the memory it has acquired for temporary use in order to function. The available memory for the app drains, and the app cannot function. If an app is frequently used or opened, a small memory leak can result in its termination. Memory leaks emerge from programming bugs, so every app needs to be tested for this issue. Memory leak testing is done by running an app on multiple devices. By doing so, testers can check the app performance on devices with different memory capabilities, and optimize the app to function effectively on each configuration.
- Usability Testing: Also known as user experience testing, this checks an app for user-friendliness. Basically, it checks ease of use and intuitiveness, aiming to provide a seamless user experience that is free of bugs and anomalies. Since the success of an app depends on the appeal of its end-to-end user experience, it is best to do usability tests with actual customers on real devices. This is the best way to understand the preferences of the target audience. Conversely, one can have skilled testers running user scenarios that mirror the behavior of actual end-users. A few pointers to keep in mind during usability tests:
- Smooth, visually appealing layout and design
- A high degree of intuitiveness
- Quick response time – Most users prefer apps that launch within 2-3 seconds after tapping the icon.
- Performance Testing: It is important to test how an application performs under various conditions. This is where performance testing comes in. It puts the device under various forms of pressure so that it does not malfunction in non-optimal conditions. A few things that performance testing should verify:
- Device performance: Installation and log-in time, battery consumption, memory consumption, etc.
- Network performance: Delays, errors, pauses in receiving digital information or rendering network-activated features
- API/Server performance: Speed and formation of data transfer from back-end to front-end
- Recovery capabilities: Built-in back-up and recovery functions that can save or recover user data in the event of data loss.
- Security Testing: App users are becoming increasingly conscious of issues surrounding data security. Online privacy and confidentiality of personal data are major concerns for most netizens – 70% report being concerned that their personal information will be shared without permission. In fact, 81% of users say they would uninstall an app and switch vendors because of security concerns. Needless to say, security testing is imperative to the success of an app. Since almost every app requires some kind of personal information to run, tests must be conducted to fortify them, in order to provide confidentiality of data. QAs must thoroughly check that the application is able to defend its users from having their information leaked or hacked in. This is especially true of financial apps.
The seven mobile testing types must be incorporated into every app testing plan in order to ensure the release of a robust, user-friendly app with the highest chance of success. Bear in mind, however, that all mobile app testing types must be executed on real devices. Whether manual or automated app testing, the involvement of real devices, and operating systems is non-negotiable. There simply is no other way to guarantee real-world success without testing in real user conditions.