Android Testing StrategyMarcelo BenitesBlockedUnblockFollowFollowingJan 10Testing an Android application was always hard.
In the past couple of years we got access to better tools, but we still rely a lot on architecture and design decisions in order to properly test an Android application.
This article will discuss the layers of automated testing an Android application should have, from End-to-end, Component to Unit tests.
Arrange, Act and Assert!Automated testing can be summarized into three steps.
Arrange: the step where we elaborate the test.
Act: when the code under test is exercised.
And finally Assert: where the result or behavior of the code is verified.
Dependency Injection is a crucial part of the Arrange step, since it allows different configurations for a specific class.
In other words, we can inject different dependencies for a specific class so when we Act, we can Assertdifferent outcomes.
In Android world there are limitations for Dependency Injection.
Developers don’t have access to Android components’ constructors since the Android OS controls the instantiation of Activity, BroadcastReceiver, Service, ContentProvider and Application classes.
There are some libraries like Dagger, or some strategies one can use to be able to inject dependencies in Android components, but there are other limitations in order to Arrange tests.
For example, if a test depends on a specific Android OS event (e.
Batter Level, Sensors, etc…), there is currently no easy way to simulate them.
Another problem is the speed of the Android emulator.
There were huge improvements in Android emulator speed, yet it is still cumbersome to do TDD in case your tests depend on the emulator.
Clean ArchitectureIn order to mitigate the aforementioned problems, a well defined architecture is necessary.
By separating the code into different layers and clearly defining borders with abstractions between them, it is possible to test parts of an Android application independently.
It is even possible to completely abstract Android SDK from the application’s use cases, so they could be reused for a different platform.
If one attempts to follow Clean Architecture by the book in an Android application it can end-up with accidental complexity, due to the multitude of classes that need to be created.
But one can use the concepts behind Clean Architecture to isolate the layers of an Android application so they can be easily tested.
One of the key concepts of Clean Architecture — if not the most important one — is the central role of the Domain.
If we divide our application into three parts: Presentation, Domain and Data.
Where Presentation is composed by Views, Fragments, Activities, Presenters, ViewModels and Controllers.
Domain is composed by all the classes that implement application use cases.
Data is a thin layer that adapts to the needs of the Domain, by providing data, for example from a Database via SQL, from an external API via HTTP or from a Bluetooth connection via Android SDK’s BluetoothAdapter.
Domain has a central role because it is unaware of any implementation details of Presentation and Data layers.
Usually Presentation will have a reference to the Domain classes, requesting the execution of use cases according to GUI events fired by user interaction.
The Domain will then request data from the Data layer through an abstraction (e.
an interface), so it will be indifferent of where the data comes from.
When the Domain is done processing the use case it will notify Presentation via another abstraction (e.
a Callback or Observable).
Flow of control — Clean ArchitectureNote that throughout the flow the Domain always used abstractions to talk to the other layers, and most importantly abstractions which are defined within the Domain itself.
If we were to create a library from the Domain classes, the interfaces that represent the abstractions would be part of the library.
Domain and Data — Clean ArchitectureUnit TestsDomain FirstOnce the application architecture establishes that the Domain layer is independent, it is easy to write Unit Tests for it.
Also since there are no dependencies on the Android SDK, the tests can run in the local JVM.
While the tests are being written for the Domain the developer can create the contract (abstraction) that the Data layer will later fulfill.
Using Test Doubles(Stubs, Fakes or Mocks) the Domain can be exercised and understood without the burden of thinking about Data implementation details.
The developer doesn’t need to focus on Presentation at first, as well.
These are the reasons why the Domain usually is the best starting point for any new feature and after it is implemented, the developer can focus its attention to Data and Presentation layers.
DataData layer should be a thin layer that will adapt a specific data source to the Domain contract.
Always be suspicious of a complex Data layer, it may be implementing business logic that should be in the Domain.
In order to help test the Data layer, typically I would go for a tool related to the technology I’ve chosen to implement it.
For example OkHttp is quite famous library in Android universe and it has a tool called MockWebServer, which basically makes it easy to mock responses and verify requests.
If we went for SQLite to implement the Data layer, SQLiteOpenHelper has an option to create an in-memory database which is perfect for testing purposes.
The key factor is that the test has to be hermetic and should not perform any I/O operations.
In that way we avoid flakiness (e.
a test failing because of network issues) and it will make the tests run way faster.
If the chosen technology has no tool to facilitate testing, we can create new abstractions to isolate the logic related to data fetching/serialization and then use Test Doubles for technology specific classes.
PresentationPresentation is a tricky part in Android, because it is difficult not to have a hard dependency on the Android SDK.
After all we use Fragments, Views and Activities to create the GUI.
We could create Presenters, ViewModels and Controllers to help abstract the Android SDK, and to be sincere that was my strategy till recent times.
Since the Google I/O of 2017 there is a push to integrate Robolectric into Espresso testing framework.
Robolectric helps in the implementation of Unit Tests by creating a Stub version of the Android SDK classes, which allows the tests to run on the local JVM.
Today it is possible to add Espresso tests to the /test folder of your Android project and they will run on the local JVM using Robolectric, instead of on an Android emulator.
It is needless to say the tests run way faster without the emulator and since that breakthrough, from a testing perspective, the importance of Presenters, ViewModels and Controllers has diminished.
Of course if a specific screen of an Android application has complicated presentation logic, those classes are helpful, but there is no need to create them for every screen anymore.
Component TestsAfter all layers of the architecture are tested in isolation, there is still need for a test that integrate all of them to verify a use case is properly implemented.
A Component test can be created with Espresso to exercise Presentation, Domain and Data through the GUI.
Component tests are going to be the ones that will allow you to sleep at night, after a major refactor of the application.
Because they are inherently Black Box tests, they don’t need to be changed when implementation details change.
Unit Tests tend to be White Box, therefore more tighten to implementation details.
If we were to elect the most important type of tests, they would be the Component ones.
A Component test has to be hermetic, therefore we should fake any external services (e.
Database or Web Service).
In order to do that, we can use the same tools and abstractions that we’ve used to test the Data layer.
Also I favor executing Component tests on an Android device or emulator, instead of using Robolectric, so we can have maximum possible classes integrated.
Fake DataDue to the lack of access to Android components’ constructors, the trick part in Component tests is to replace parts of the Data layer with Test Doubles.
The first thing is to separate the classes that provide dependencies from the classes that actually use them.
Dagger helps to accomplish that, but there are other ways, like using the Application as a dependency provider.
Actually, for Dagger to work properly, the Application has to provide a Dagger Componentanyways.
Once we have a centralized place that provides the dependencies for the application, there are several ways to replace some of them for testing purposes.
One is to use a custom AndroidJunitRunner to replace the entire Application only when running the tests (it would allow us, for example, to replace a Dagger Component).
Another way is to use reflection to replace specific dependencies with Test Doubles.
The former is easier but it replaces the entire Application removing a part of the code that should be under test.
The latter is more complicated to implement, but it changes less code, therefore resulting in a more realistic test suite.
End-To-End TestsFinally End-To-End tests are the ones that exercise the full stack of an application, including external services.
End-To-End tests are not hermetic and are flaky by default, thus they can provide false negatives.
When such tests fail, their results should be double checked by a human being to verify that the reason is not, for example, due to intermittent internet connection.
Also End-To-End tests are hard to maintain because their Arrange step is complicated.
Sometimes it means setting up an isolated environment (e.
Staging or QA environment), or calling APIs to prepare the state of the system before the tests run.
Because of the aforementioned issues, an application should have few End-To-End tests.
Usually they are focused on the critical paths of the user in the application, therefore error states and edge cases should not be part of an End-To-End test suite.
ConclusionIn order to throughly test an Android application different types of automated tests are necessary.
By missing one of those types the application becomes vulnerable and the confidence the developers have on changing it diminishes.
Unit Tests serve to drive the implementation (specially when TDD is being used), Component Tests verify the use cases are properly implemented and finally End-to-End Tests verify the interaction of the application with the rest of the system.
Testing Strategy — Summary.. More details