Test from the Command Line

This document describes how to create and run tests directly from the command line. This document assumes that you already know how to create a Android application in your programming environment.

Running Tests

You can run tests from the command-line, either with Gradle or with an Android Debug Bridge (adb) shell.

The Android plugin for Gradle lets you run unit tests from your Gradle project via the command-line. For more information on how to build unit tests for your app, see Building Effective Unit Tests.

Running unit tests with Gradle

The Android Plugin for Gradle lets you run unit tests from your Gradle project via the command-line. For more information on how to build unit tests for your app, see Building Effective Unit Tests.

The table below summarizes how to run your unit tests with Gradle:

./gradlew test

./gradlew connectedAndroidTest

Gradle supports task name abbreviations. This means, for example, you can initiate the connectedAndroidTest task by simply entering the following command.

./gradlew cAT

The test and connectedAndroidTest tasks run tests on each module and build variant in your project. You can run tests for just a specific module in your project by prefixing the test or connectedAndroidTest task with the module name and a colon (:). For example, the following command runs instrumented unit tests for just the mylibrary module.

./gradlew mylibrary:connectedAndroidTest

You can also target a specific build variant using the following syntax.

• For local unit tests:

  ./gradlew testVariantName
  

• For instrumented unit tests:

  ./gradlew connectedVariantNameAndroidTest
  

Note: If you don't specify a target module to test, Gradle looks through all your modules and runs tests for each variant that matches the configuration name you specify.

Gradle also allows you to target specific tests using the --tests flag. For example, the following command runs only the sampleTestMethod tests for the specified build variant. To learn more about using the --tests flag, read Gradle's documentation on test filtering.

./gradlew testVariantName --tests *.sampleTestMethod

Multi-module reports

As described in table 1, Gradle saves test reports in the build/ directory of each module that it tests. However, when running tests across multiple modules, it may be useful to combine all the test results into a single report. To generate a single report when running tests across multiple modules, proceed as follows:

1. In your project-level build.gradle file, add the following after all other configurations in the file.

   apply plugin: 'android-reporting'
   

2. Invoke the test or connectedAndroidTest task with the mergeAndroidReports task. For example:

   ./gradlew connectedAndroidTest mergeAndroidReports
   

   If you want to skip test failures in order for Gradle to finish running all remaining tests, add the --continue option:

   ./gradlew connectedAndroidTest mergeAndroidReports --continue
   

When Gradle finishes running all your tests, it saves the combined reports in the PATH_TO_YOUR_PROJECT/build/ directory.

Running tests with ADB

When you run tests from the command-line with Android Debug Bridge (adb), you get more options for choosing the tests to run than with any other method. You can select individual test methods, filter tests according to their annotation, or specify testing options. Since the test run is controlled entirely from a command-line, you can customize your testing with shell scripts in various ways.

To run a test from the command-line, you run adb shell to start a command-line shell on your device or emulator, and then in the shell run the am instrument command. You control am and your tests with command-line flags.

As a shortcut, you can start an adb shell, call am instrument, and specify command-line flags all on one input line. The shell opens on the device or emulator, runs your tests, produces output, and then returns to the command-line on your computer.

To run a test with am instrument:

1. If necessary, rebuild your main application and test package.
2. Install your test package and main application Android package files (.apk files) to your current Android device or emulator
3. At the command-line, enter:

   $ adb shell am instrument -w <test_package_name>/<runner_class>
   

   where <test_package_name> is the Android package name of your test application, and <runner_class> is the name of the Android test runner class you are using. The Android package name is the value of the package attribute of the manifest element in the manifest file (AndroidManifest.xml) of your test package. The Android test runner class is usually AndroidJUnitRunner.

   Your test results appear in STDOUT.

This operation starts an adb shell, then runs am instrument with the specified parameters. This particular form of the command will run all of the tests in your test package. You can control this behavior with flags that you pass to am instrument. These flags are described in the next section.

Using the am instrument Command

The general syntax of the am instrument command is:

am instrument [flags] <test_package>/<runner_class>

The main input parameters to am instrument are described in the following table:

The flags for am instrument are described in the following table:

am instrument options

The am instrument tool passes testing options to AndroidJUnitRunner or InstrumentationTestRunner in the form of key-value pairs, using the -e flag, with this syntax:

-e <key> <value>

Some keys accept multiple values. You specify multiple values in a comma-separated list. For example, this invocation of AndroidJUnitRunner provides multiple values for the package key:

$ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 > com.android.test/android.support.test.runner.AndroidJUnitRunner

The following table lists the key-value pairs you can use with your test runner.

• am instrument invokes onCreate(Bundle) with a Bundle containing the key-value pairs.
• The package key takes precedence over the class key. If you specifiy a package, and then separately specify a class within that package, Android will run all the tests in the package and ignore the class key.
• The func key and unit key are mutually exclusive.

Usage examples

The following sections provide examples of using am instrument to run tests. They are based on the following structure:

• The test package has the Android package name com.android.demo.app.tests
• Two instrumented test classes:
  • Foo1 which contains the test method bar1, and
  • Foo2 which contains test methods bar2 and bar3
• The test runner is AndroidJUnitRunner.

Running the entire test package

To run all of the test classes in the test package, enter:

$ adb shell am instrument -w com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner

Running all tests in a test case class

To run all of the tests in the class UnitTests, enter:

$ adb shell am instrument -w  > -e class com.android.demo.app.tests.Foo > com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner

am instrument gets the value of the -e flag, detects the class keyword, and runs all the methods in the UnitTests class.

Selecting a subset of tests

To run all of the tests in Foo1, and the bar3 method in Foo2, enter:

$ adb shell am instrument -w > -e class com.android.demo.app.tests.Foo1,com.android.demo.app.tests.Foo2#bar3 > com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner