🐘 How to add consistency between local and CI environments when running Android instrumented tests?

🧐 The Why

Hi everyone, ever wondered your tests run locally without any issues but fail on the CI(Continuous Integration) server? This can have many reasons such as different local environment compared to the CI environment. For example if you’re using an M1 Mac to develop your Android app vs. an x86 linux environment on the CI. Or maybe different emulator versions, especially the emulator version on the CI can get old and someone might forget to update them.

In November 2022, the Gradle managed devices were introduced for Android. It hopefully would make running tests more reliable by adding consistency between local and CI environments.

The Gradle task will simply take over anything related to creating the emulator, running the instrumented tests, and providing the test results in the build folder. The task will even automatically detect the CPU type of the environment and sets up the device accordingly, that means same config can run on a local ARM environment and on a x86 CI environment without changes. This is all simpler, requires less configuration, and gradle tasks are more familiar to Android developers. Also it is supported by the Android team at Google.

πŸ› οΈ The How

All one has to do to configure a Gradle-managed device, is to add the device in the module’s build.gradle file:

android {
  testOptions {
    managedDevices {
      devices {
        maybeCreate<com.android.build.api.dsl.ManagedVirtualDevice>("pixel2api31").apply {
          device = "Pixel 2"
          apiLevel = 31
          systemImageSource = "google"

Now, if you have a build variant named .debug, you can run the Android instrumented tests for it all in one line:

./gradlew pixel2api31DebugAndroidTest

This will run a headless Android emulator from the command line, runs your Android instrumented tests on it.

πŸ“ How to export the coverage file from the managed device?
It’s very simple!
Once the task is finished, you can find the output files such as your test coverage files (e.g. Jacoco .xml or .ec coverage files) under the following path:


// for example, if you have enabled jacoco to produce an .ec coverage file, it will be at the following path:

🎁 Bonus

If you are working with a CI system with MacOS environments and intel CPUs, or you just want to simulate it on your local environment, the following bash script can silently install android sdk, accept the licenses, install the intel HAXM for virtualisation and run the instrumented tests on a Gradle managed device:

# install android commandline tools and required packages
brew install --cask android-commandlinetools
yes | sdkmanager "tools" "platform-tools" "build-tools;33.0.1" "platforms;android-33" "system-images;android-31;google_apis;x86_64" "extras;intel;Hardware_Accelerated_Execution_Manager"

# accept licenses
yes | sdkmanager --licenses
export ANDROID_HOME=/usr/local/share/android-commandlinetools
export PATH=$PATH:$ANDROID_HOME/tools:$ANDROID_HOME/tools/bin:$ANDROID_HOME/platform-tools

# install intel haxm
cd $ANDROID_HOME/extras/intel/Hardware_Accelerated_Execution_Manager/
chmod +x silent_install.sh
sudo ./silent_install.sh
/usr/local/share/android-commandlinetools/tools/emulator -accel-check
cd ~/project/

# run the instrumented tests using gradle managed devices
./gradlew pixel2api31DebugAndroidTest

More Info about using Gradle Managed Devices on Android Official Docs.

Please leave a comment for your questions and feedbacks. πŸ™‚