Testing with emulator.wtf

How to run tests using emulators from emulator.wtf

Configuring emulator.wtf API token

In order to use emulator.wtf service for app testing, you need to obtain an emulator.wtf API token and save it as an environment variable in Codemagic.

1. Open your Codemagic app settings, and go to the Environment variables tab.
2. Enter the desired Variable name, e.g. EW_API_TOKEN.
3. Copy and paste the content of the token as Variable value.
4. Enter the variable group name, e.g. emulatorwtf. Click the button to create the group.
5. Make sure the Secure option is selected.
6. Click the Add button to add the variable.

You can then import the token in your workflow:

  workflows:
    android-build:
      environment:
        groups:
          - emulatorwtf # adds EW_API_TOKEN to the workflow

Running tests

Add the following snippet under your workflow after the Gradle build step. Update the --app and --test paths according to your build if necessary.

scripts:
  - name: Test
    script: | 
      ew-cli \
        --app app/build/outputs/apk/debug/app-debug.apk \
        --test app/build/outputs/apk/androidTest/app-debug-androidTest.apk \
        --outputs-dir results
    test_report: results/**/*.xml

Capturing logcat

Add the following to your workflow to capture logcat output from the emulator during the test run:

  artifacts:
    - results/**/logcat.txt

Running tests with coverage

Add --with-coverage to the ew-cli script to capture coverage during the test run:

scripts:
  - name: Test
    script: | 
      ew-cli \
        --app app/build/outputs/apk/debug/app-debug.apk \
        --test app/build/outputs/apk/androidTest/app-debug-androidTest.apk \
        --with-coverage \
        --outputs-dir results
    test_report: results/**/*.xml

Running tests in parallel

Add --num-shards <NUMBER> to run tests in parallel shards, here’s an example to shard tests to 4:

scripts:
  - name: Test
    script: | 
      ew-cli \
        --app app/build/outputs/apk/debug/app-debug.apk \
        --test app/build/outputs/apk/androidTest/app-debug-androidTest.apk \
        --num-shards 4 \
        --outputs-dir results
    test_report: results/**/*.xml

Running tests with orchestrator

Add --use-orchestrator to run tests with Android Test Orchestrator (more infos here):

scripts:
  - name: Test
    script: | 
      ew-cli \
        --app app/build/outputs/apk/debug/app-debug.apk \
        --test app/build/outputs/apk/androidTest/app-debug-androidTest.apk \
        --use-orchestrator \
        --outputs-dir results
    test_report: results/**/*.xml

Further information

There are more options available like pulling directories from the emulator after tests have finished, running on various device models, etc. Check the emulator.wtf docs for more ew-cli options to customize your test run.