Run tests in CI

Outcome: repeatable Android checks that report failures and preserve enough evidence to diagnose them.

Runner prerequisites

Your runner needs Olef, Bash for generated test scripts, the project’s build dependencies, and access to a running emulator or device. Local builds require the Android SDK and JDK. Provision the runner’s emulator before these steps; this example does not create one.

Install your app build before running recorded tests. Configure the app package and target expected by the test. Use development accounts and data.

Run the suite

From the project root on a prepared runner:

olef --version
olef devices
olef connect
olef test run tests/olef --junit-out build/olef/junit.xml --artifacts-dir build/olef/runs

The test runner returns a non-zero exit code on failure. Preserve that exit status in CI. Upload build/olef/ as an artifact even when the command fails, and publish the JUnit file using your CI provider’s test-report feature.

Check repeatability

olef test run tests/olef --repeat 3 --artifacts-dir build/olef/repeats

Inspect stable failures and flaky results separately. Retries help identify intermittency; they do not turn a flaky behavior into a stable pass.

For parallel workers, use separate devices and explicit targets. --shard I/N partitions the test list; it does not provision or isolate devices by itself. See sessions and test runs.

Existing instrumentation tests

Olef’s connected service occupies Android’s UiAutomation slot. Wrap tests that need the slot:

olef test -- ./gradlew connectedDebugAndroidTest

The wrapper disconnects Olef, runs the command, and reconnects. This is distinct from olef test run, which runs Olef test folders.

Cloud runners

Olef Cloud supports remote workflows subject to account access and limits. Use Cloud setup and the cloud reference for authentication and workspace lifecycle. Inject credentials through the CI secret store; do not commit access keys or session tokens to a test folder.

Ask your agent

Integrate the existing Olef tests into this repository's CI. Inspect our
current build and emulator setup first. Preserve test exit codes, publish
JUnit, and upload diagnostic artifacts on failure. Do not hide failing
checks with unconditional success or retries.