Creating automated tests on Android is quite a difficult procedure. To manage the process of automated testing correctly, one needs to plan and analyze a lot of tasks.

But the greatest difficulty lies in the fact that on the Internet there’s no detailed description of what exactly includes the automation process for the Android system, what are its basic stages, etc. Unfortunately, there’s simply no overall pattern.

However, before we will analyze available software and methods of its usage, we have to figure it out – why do we need automated tests in the first place?

What Is the Relevance of Autotests?

There is a notion that UI tests aren’t relevant if there is enough number of unit tests and integration checks on a project. Nevertheless, the following metaphor will provide insights into this situation.

Let’s imagine that we are bicycle engineers. We have two thoroughly tested wheels, a frame, a saddle, and a handlebar. If we speak using QA terminology, we have a set of unit tests and a group of integration tests.

A question is: will the bicycle go now? To answer this question we have to involve a group of manual testers. Before every future release, they have to assure that ideal details are perfectly interacting with each other. Then, our bicycle will go correctly giving its owner real pleasure.

Something similar we can notice about software for mobile application testing. Unfortunately, in the modern mobile world there is no opportunity to quickly cancel unsuccessful editing since all the versions go through App Store / Google Play.

Hence, it immediately put particular constraints like long loaded covering in comparison with desktop web products. That’s why it is significantly important before final releases to ensure that basic user scenarios function exactly as originally agreed.

Choosing the Correct Tool To Write Mobile Automated Tests

First of all, we need to define one the only sure technology stack that is necessary for the process of automated test creation.
The first case structure is the choice between cross-platform solutions and native ones. On the Internet, people always argue about what is relevant and what doesn’t need proper results anymore. Although, we lean towards native solutions because they are:

• More stable;
• Fast;
• Qualitatively integrated into IDE;
• And don’t have layers that may cause flapping and suggest extensive technical knowledge.

Moreover, Google supports Espresso and UI Automator. Today there are really few people who write tests on Espresso only. In recent times, programmers developed a lot of useful solutions for current issues.

As a consequence, on the Internet, there are a lot of useful frameworks and programs that make mobile testing on Android simple and straightforward. Let’s take a look at them.

Kaspresso

This framework is useful because it can:

• Help to manage DSL which significantly simplifies the creation of autotests;
• Provide multi-level protection against flaky tests;
• Extremely improve the functioning of UI Automator;
• Provide full logs about what is going on during a test run;
• Allow user to run any ADB commands inside a test;
• Describe the best practices on autotest creation.

Test Runner

Imagine that we have several tests. And we need to run them. The test runner is responsible for this stage.

We can find some solution in Google. For example, we can use such a utility as AndroidJUnitRunner and its special mode Orchestrator. This utility simply does what is required: it runs a set of autotests allowing do it in parallel.

Orchestrator allows specialists to keep running the tests even if some of them are failed. Also, this mode can minimize the general technical state among running tests. In such a way, testers reach the general isolation of each test run.

Sometime later, more and more requirements are put for test runners. For example:

1. Run separate groups of tests;
2. Run automated tests on particular mobile platforms and devices;
3. Show test results;
4. Save history of test runs for future analysis;
5. Integrate tests with specific local infrastructure.

Nowadays, on the Internet, we can find a lot of test runners. The most popular of them is Marathon that can meet some requirements described above. For example, this tool can support parallel test execution and preparation of test results in the form that can be described in Allure.

But, unfortunately, Marathon doesn’t have such features as:

1. Simple integration of test runner with a technical infrastructure that can give emulators;
2. Solid integration with frameworks (for example, Kaspresso) to get the most accurate data about a test.

Moreover, there is a perception that a test runner must be platforming for both Android and iOS. This is caused by the peculiarities of every mobile operating system and the resultant criticality of modification implementation into a test runner.

Where To Run Automated Tests?

The answer to this question can be the three following groups:

1. Real devices. One of the advantages is that usage of real devices allows you to find problems specific to particular gadgets and their firmware. Sometimes it’s very useful to ensure that the developed application is working properly on the device you’re testing it on.
   A disadvantage is that you always have to find a lot of devices, make special space for them. Besides, some tests can change the device state, and often it is really difficult to get back from the broken version to the last stable one.
2. Emulators. It is faster and more stable than a real device. The development of a new emulator takes just a couple of minutes.
   On the negative side, there is a limited number of test scenarios that will be based on peculiarities of a specific device, and not all of them will be of highly-priority.
   But the biggest disadvantage is a low percentage of scalability. A simple task to upload a new version of the emulator to a particular host is a problem in itself.
3. Docker version of Android emulator. Docker and rollout of emulator image is a great scalable solution that allows quickly creating emulators and rolling them out on hosts reaching sufficient isolation. However, it has a disadvantage – the highest input threshold.

Interaction With Infrastructure

Let’s imagine that we’ve created a couple of hundreds of UI tests. It is a huge number but they need to be run very quickly (for example, maximum in 20 min). Here, we come to real scaling.

This sphere is an unknown topic for some Android developers and most automated QA. And it’s not surprising because such infrastructure demands the highest knowledge of equipment, the configuration of a server functioning, as well as DevOps practices.

Hence, it is extremely important to establish the right contact with people who are professionals in this sphere. It can be both embedded contractors and specially trained employees inside your office.

In a few words, here’s what you’ll need to do:

1. Choose between cloud and local solutions provided that the company has a personalized infrastructure for starting tests for different platforms.
2. Deploy the whole internal infrastructure from the beginning. With such work, you need to enlist high-quality and proven equipment that will withstand the maximum number of autotests you run. We’ll have to analyze the work of several simultaneously running emulators and monitor the stability of the tests.
3. In the end, be sure to operate with some service that gives you emulators. This can be, for example, Kubemetes – a cloud solution similar to Google Cloud Engine or a custom solution.
4. Combination of the received service with Test Runner.

Summary

In conclusion, we want to note that you should not forget to:

• Extract report after the execution of the tests;
• Perform synchronization with TMS;
• Integrate into CI / CD;
• Train programmers and testers in methods of creating and checking such autotests;
• Fix the processes of creating, changing, and using mobile autotests.

In all of the above, we tried to describe as fully as possible the basic stages of the formation of the auto testing process for the Android mobile platform. We strongly believe that after reviewing this material, the reader will have an exceptionally correct view of this situation as a whole.