Top Challenges Your Mobile App Testing Will Face With Android O Release

Google has announced the availability of the initial developer preview release for its upcoming new major OS release – Android O (Rumored to be named “Oreo”).

While this is only a developer preview and some of the included features in it can be pulled out in the future, and things can change – from a Dev perspective, this should be a perfect time to assess the implications such a release might have on your existing app quality, as well as the new opportunities underneath this major release.

Before drilling into the Android O release content, let’s understand the current Android market landscape.

As can be seen above, the latest Android N release is less than 3% market share with Samsung lead devices, and LG V20 is catching up with this version to perhaps drive that share toward the 5-7% soon.

The major market grab is still thanks to Android M and L, that together account for ~60% of the Android market.

Even though Google pushes through innovation in the market in an attempt to increase the adoption of its latest OS releases, there won’t be a significant revolution during 2017. That means fragmentation is still growing within the Android landscape.

By fragmentation, it’s important to understand that this does not only mean more devices and OS combinations in the market to test against – it also means that from both supported features and test cases, there should be different branches that tie the test suites to a device/OS combination and a supported capability.

It’s a Growing Problem

As just one example, I took my personal Samsung Galaxy Note 5 – with its week-old Android N (7.0) OS upgrade – and tried leveraging a new feature that was introduced as part of that OS called App Shortcuts.

Amazingly, when using this feature on the Twitter app, although it works great and allows me to write a new Tweet from the home screen by pressing on the app icon and selecting that menu option, when trying to interact with the Google Photos native app through the new app shortcuts, the app constantly crashes (see below screenshot). It is important to state that this feature works fine on that app on the Google Pixel (Android 7.1.1 OS) device. This is just one of many fragmentation problems from a testing perspective. Quite simply, there is a growing gap between test suite, app under test, and the target platforms under test.

When considering the current market state and its constraints, let’s complicate things with what’s expected in Android O.

Android O Capabilities: The Highlights

• App Background Limits: With the Android O release, developers can enforce more limits on the app resource usage to save battery and other HW device resources. This capability can allow an app based on pre-defined limits to better manage the device battery while in the background, through limits on the use of location services and signal activities.
  • The impact of this new capability on the Dev and Test teams, assuming they are implementing that capability, is obviously more new tests and branching of these tests from Android < O releases. Within these new test cases, there will need to be tests to assure that there are limitations based on the location constraints set by the developers, and other limits that are supported.
• Notification Grouping: This new capability will allow users to group the notifications by types and to control which notification comes from which app, together with other criteria, so they can better manage the “noise” they receive.
  • The impact of this feature would again be that more testing would need to be added to support the granular notifications that will be supported, as well as the configuration aspect on the device, and environment-related testing that will need to happen around the notification bar when other popups, notifications, and events occur.
• AutoFill API: This is a usability innovative feature that can allow users to set an “AutoFill” app that can support a specific need like password management, etc., without having the user launch that app in parallel.
  • The impact of such features can vary based on the app context that is being developed since each app will have external needs and sometimes more than one (password management is only one example). Testers would need to understand which apps can serve as AutoFill for their AUT (app under test) and test against them. In that case, it will also make sense to leverage scenarios that were mentioned in bullet 2 like environments, system events, etc.
• Adaptive Icons: Here, developers can develop the app icons in ways that will look different based on the platform they are installed and launched on.
  • This opens a range of visual-based test automation scenarios that will require a set of different devices with different resolutions and perhaps themes to assure that the dev animated icons look and move/adapt as expected.

Bottom Line

In this post, we examined a current market landscape that is already fragmented from both device and OS perspectives. That produces great challenges for Agile teams around test automation optimization. We touched on the upcoming Android O release and tried to identify some immediate insights and implications for current teams that develop and test Android native apps. We’ve left for another day the issue of Bluetooth audio quality enhancement, which is clearly another disruptive move in the mobile and mobile web space that both Dev and Test engineers ought to be ready for – leverage the early versions from Dev, EA, and Betas to be prepared for the launch. Are you ready?