Set Up Your Continuous Testing Process for Android
Set Up Your Continuous Testing Process for Android
Android testing can be painful but not if you have the right tools and processes in place.
Join the DZone community and get the full member experience.Join For Free
Test automation is an indispensable part of any software development process, including mobile app development. Trying to run all your tests manually simply takes up far too much time, time that could be better spent on improving your code, or, well, a bunch of other things that are far more useful than running unit tests one by one. Of course, 100% automation is impossible: you simply cannot eliminate all the tedium from testing. Sorry.
You may also enjoy: The difference between a DevOps and a Mobile DevOps lifecycle
The Need for CI in testing
Automating as much as humanly possible makes developers' lives a lot easier. As such, continuous integration services must provide the tools that allow users to run automated tests with a minimum of fuss and then analyze the test results afterwards. Let’s look into one of those services that can do all that: Bitrise.
Bitrise is a CI/CD service with a strong focus on mobile app development: you can build any kind of software on it but it is geared and serviced with mobile applications in mind.
It is based on so-called Steps, which are build tasks that each deal with a certain part of the continuous integration process: they download dependencies, build your app, deploy it, and of course run tests.
Building a Testing Process
The flexibility of the platform means that when it comes to running tests on Bitrise, everything is up to you: you can create your own process, using only your very own custom scripts.
However, Bitrise offers plenty of “pre-packaged” Steps to do the job for you — let’s take a look at some of those, using Android as our example. But similar Steps to the ones we’ll be talking about exist for most platforms, including, of course, native iOS, but also Flutter, React Native, or Xamarin.
Now let's dive in!
Android Testing 101
The open nature of the Android system and the myriad of available devices potentially make Android testing a bit of a nightmare: careful planning of your testing process is vital. CI tools do not take this burden off your shoulders, nor do they write your tests for you, but they provide tremendous help in executing your processes.
Prevent Structural Problems
First and foremost, you’ll want to ensure your code has no structural problems. To do so, you have to run the code through lint — strictly speaking, this isn’t testing as the concept is generally understood but it’s definitely part of the process of ensuring the quality of your app. And running lint on Bitrise is the easiest thing in the world: take your Workflow, and add the Android Lint Step to it on the graphical UI.
In the Step’s inputs, you can set the module and the variant you want to check or you can leave those empty to run lint through everything.
Since the Step will, essentially, run a Gradle task, you can append extra arguments passed on to that task in the Options section of the Step’s input. In the same section, you can restrict linting to certain files or directories if you wish.
Integrate Your Unit Tests
You probably also have some unit tests in your app, to make sure each part of the code functions as it should. Of course, these can be executed on Bitrise, too: just add the Android Unit Test Step to your Workflow.
It will locate any unit tests in your repository, and execute them if the Step’s configuration allows: as with Android Lint, you can restrict the Step to testing certain modules or variants only, and set additional Gradle arguments.
Test on Device
Finally, you’ll obviously want to test your app on some Android devices before publishing it. One way to do it is to simply get a bunch of Android gadgets, and install the app on them. This is, to put it mildly, a little time-consuming and not very efficient.
Bitrise offers an easier way out: the Virtual Device Testing for Android Step. The Step runs the tests on Firebase, and you don’t need your own Firebase account to use it so there’s no need to provide credentials or anything of the sort.
It’s very simple: you write your tests, and on Bitrise, you create a Workflow that builds your app for testing — look into the Android Build Step — and add the Virtual Device Testing for Android Step to it. It can run either of three test types, and you can configure them further in the options below them:
Set the test type, and the device type and OS version, as well as language and orientation. Obviously, you are restricted to the devices available on Firebase but within those, you can run your tests on as many devices as you like, with as many different configurations as available. The choices aren’t quite endless but for most developers' needs, they’ll do just fine.
See All Your Test Results in One Page
Once the tests have run, someone will have to take a look at the results, to see if everything worked out quite as well as the developers intended. Scouring the build logs for test results is a bit of a pain: there are probably a lot of other data in that log, the output of several different build tasks. For this, you can just use the Test Reports add-on.
To use it, you need to do one thing: include a Deploy to Bitrise.io Step at the end of your Workflow. This Step performs a number of tasks but for now, the important thing is that it exports your test results to the add-on.
Note that for it to do so, the results must be in the appropriate directory, in a supported format: some Steps, such as Android Unit Test or Xcode Test for iOS, automatically package their results the right way, while for other Steps, you need to use the Export Test Results for the Test Report add-on Step to have their results available in the add-on. For details on this, check out Bitrise’s official documentation.
You can enter the add-on from the build page of your app, and once you’re in, you will see all your tests, grouped by their name, with their success or failure highlighted for you. You can check out all the logs here, as well as view and download screenshots and videos of UI tests.
Test automation is essential to a successful software development process.
Careful planning of your testing process is vital.
Make sure your code has no structural problems before you actually start testing.
Save time by integrating unit tests and tests on device seamlessly with out-of-the-box Bitrise steps.
Finally, don’t forget to analyze your results!
Opinions expressed by DZone contributors are their own.