Testing on AWS Device Farm
How to configure Flutter Driver tests in the Flutter workflow editor
- An IAM user with Device Farm permissions is required. Follow the instructions outlined in the setup document to create an IAM user with access to Device Farm and make note of the access key ID and secret access key.
sylph.yamlfile in the repository for AWS Device Farm configuration (optional). The
sylph.yamltemplates are available on GitHub.
AWS Device Farm setup on Codemagic
- Navigate to App settings > Test.
- At the top of the section, select Enable Flutter Driver tests to enable running integration tests for this workflow.
- Select to run Flutter Driver tests on AWS Device Farm.
- Enter your AWS access key ID and AWS secret access Key to the respective fields.
For testing on iOS devices, additional configuration is needed. Follow the instructions in the iOS builds section in Sylph readme and add the required environment variables to Codemagic in App settings > Environment variables.
During the build, Codemagic searches the repository for a
sylph.yaml file to configure the AWS Device Farm test run and generates a default configuration file if no existing configuration is found. By default, we run the tests in your
project_root/test_driver folder on Google Pixel 2 (OS 8.0.0) regardless of the build platforms you have selected.
After the test run, you will see an overview of how many tests failed, passed or were skipped on the Results tab as well as a detailed test log containing information about used device minutes on the Log tab.
Changing the Flutter version used for testing
Sylph comes with a predefined Flutter version set here and doesn’t yet support version changing.
If you want to test your application with another Flutter version, follow these steps:
Fork the Sylph repository and change the Flutter version in
test_spec.yamlto the desired one. Note that the version name is followed with the channel name.
In Codemagic, add a pre-test script to activate your fork.
If you made your changes to the forked repository on the default branch, you can simply mention the git URL to activate it:
pub global activate -s git https://github.com/your_username/sylph.git
If you made your changes on a specific branch e.g.
flutter_1.17.5, you can clone the specific branch and specify the path to the cloned repository:
git clone --branch your_branch_name https://github.com/your_username/sylph.git $HOME/sylph pub global activate -s path $HOME/sylph