The different dimensions of Mobile App Testing

Mobile apps have become an integral part of our everyday life. It goes without saying that successful mobile apps are those that are appealing and useful for their end-users. In particular, successful mobile apps meet a tangible need of end-users i.e. they solve a real-world problem. At the same time, they are acceptable to end-users in terms of user-friendliness, aesthetics, and ease of use. Hence, user experience ends up being an integral part of the usefulness and acceptability of the app.

In order to ensure that an app provides the right user experience, it has to be tested thoroughly. Mobile app testing is also indispensable when it comes to evaluating non-functional aspects like performance and scalability. However, mobile app testing is a challenging process, as it is associated with many different aspects and requires mastering different tools and techniques.

Testing User Interface Aspects

All successful software applications offer an acceptable and friendly User Interface (UI). Mobile apps are no exception to this rule. Hence, UI testing is probably the most common activity of a test plan for a mobile app. UI testing audits the appropriateness and acceptability of aspects like the color scheme of the device, the UI theme used, the style of the icons used, the color of the icons, the use of appropriate UI indicators and controls (e.g., progress bars), the aesthetics and utility of menu items, the overall responsiveness of every UI element, as well as the ease of navigation and use of the various commands of the mobile app.

A significant part of UI testing concerns the evaluation of how the app behaves in cases of different orientations and resolutions of the screen. Testing under different screen resolutions is important in order to ensure that the mobile app behaves appropriately across a variety of devices and network conditions. Typically, all common screen resolutions) should be tested. In case of multi-column or other complex table-like layouts, testing needs also to check that columns and tables align correctly, especially in cases of lower resolutions where scrolling is also required.

Non-Functional Tests

Mobile app testing involves many other aspects beyond UI and usability testing. In many cases, these aspects concern the non-functional aspects of the apps such as:

• Performance testing, which focuses on testing whether the app provides decent latency under various usage scenarios and in cases where users have older or primitive devices.
• Testing of cloud services, which is devoted to testing the responsiveness and the use of cloud services used by the mobile app, given that these services are in most cases critical for the app’s smooth operation.
• Internationalization testing, which aims at ensuring that the mobile app is decent in all the different languages and locales that it supports.
• Installation testing, which ensures that the installation of the application is performed properly in all the different supported devices.

The role of Mobile Devices in Testing

As evident from the description of the above testing aspects, successful mobile apps need to install and run properly across a variety of different devices. Thus, testing behavior across different devices is an essential part of mobile app testing. Device testing is a very demanding and resourceful task given the very large number of mobile devices vendors and types, including smartphones, tablets, personal digital assistants, book readers and more. To restrict the number of devices to be tested, specific devices can be prioritized based on the target audience of an app. The target audience could be defined based on mobile/web analytics, which provides insights about the number of mobile users who visit t a site or download an app, as well as about the devices and operating systems that they use.

In terms of mobile platforms and operating systems, apps need to support and be tested at least for the two most prominent mobile platforms, namely iOS and Android.  There are several testing frameworks that enable the tester to develop the required test cases for the application. A few of the popular testing frameworks are listed below.

Mobile Testing Frameworks

Mobile testing frameworks boost testing automation, by enabling development teams to build and automatically execute test cases over them. As already outlined, mobile testing frameworks cover the most popular mobile platforms. For example, some popular testing frameworks for the Android platform are:

• Robotium is an open source test framework that enables the development and execution of powerful, robust and automated UI test cases for Android apps. Based on Robotium, testing engineers can develop functional, system and acceptance test cases for Android applications.
• Selendroid offers an API (i.e. the Selenium 2 client API), which enables developers and testers to write and automate test applications involving UI elements for the Android platform, as well as for mobile web applications. Users of Selenium (i.e. of the popular portable software-testing framework for web applications) can adopt Selendroid based on a very short learning curve, as Selenium knowledge can be used to write Selendroid tests.
• UIAutomator is Google’s test framework that enables testing of UIs based on the development of automated functional UI test cases, which can be executed for a mobile app on one or more devices. It is quite widely used by game developers and comes with Java API for creating UI tests and an execution engine for running them.
• Appium is an open-source test automation framework for native, hybrid and mobile web apps. Appium automates mobile apps from any language and any test framework through providing full access to back-end APIs and databases from the test code. In particular, the framework comes with library functions that call functions on the Appium servers where the connected devices operate.
• Calabash is another testing framework that can be used for multiple mobile platforms, including iOS and Android. It’s really easy to use and provides visual interfaces and tools for non-technical users.

As evident from the description of the above-listed tools, some of them (i.e. Appium, Calabash) can be used for testing iOS applications. However, there are also exclusive iOS testing frameworks such as:

• UI Automation which is a tool is developed by Apple in order to automate the testing of iOS apps. It is very similar to Google’s UI Automator for Android, which has been described earlier. UI Automation tests are written in JavaScript based on an API defined and supported by Apple.
• Zucchini which is another iOS open-source testing framework that provides visual functional testing functionalities. It leverages the lower level capabilities of Apple UIAutomation.

No mobile app can be successful without thorough and systematic testing. However, mobile app testing is in several cases much more challenging than testing of conventional software apps, given the need to deal with multiple testing aspects and a wide range of supported devices. Therefore, successful testing lies not only in taking advantage of the right automation frameworks and tools but also in creating an effective testing plan that covers the vast majority of possible cases and ensures that all the important user requirements are successfully met.