Add nightly Maestro E2E workflow (#11181)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -29,6 +29,49 @@ adb reverse tcp:3000 tcp:3000
|
||||
- In a second tab, run `pnpm e2e:build`
|
||||
- In a third tab, run `pnpm e2e:run __e2e__`
|
||||
|
||||
## Nightly Maestro CI
|
||||
|
||||
The `Nightly Maestro E2E` GitHub Actions workflow runs every day at 04:00 UTC
|
||||
and can also be started from the Actions tab with **Run workflow**. It runs iOS
|
||||
and Android concurrently, but each platform runs all of `__e2e__/config.yml`
|
||||
sequentially on one explicitly selected simulator or emulator. The flows share a
|
||||
stateful mock-server manager, so the suite must not be sharded.
|
||||
|
||||
The jobs run Maestro CLI 2.6.1 locally on GitHub Actions; Maestro Cloud is not
|
||||
used. iOS runs on `macos-26-xlarge` with Xcode 26.4. Android runs on
|
||||
`Linux-x64-32core`. Both use Java 17 and the Node and pnpm versions declared in
|
||||
`package.json`. The iOS job selects an iPhone 17 simulator running iOS 26.5;
|
||||
Android directly provisions and boots a Pixel 6 AVD with the API 35 Google APIs
|
||||
x86_64 image using the Android SDK command-line tools.
|
||||
Both development clients use the `e2e` EAS profile and the same reusable local
|
||||
EAS build action as the release build workflows; the resulting simulator app
|
||||
and APK are installed directly on the selected devices.
|
||||
|
||||
The mock-server manager listens on host port 1986 and creates test services on
|
||||
port 3000. Metro listens on 8081. Android reverses ports 3000 and 8081 into the
|
||||
emulator; port 1986 remains host-side because Maestro JavaScript calls it from
|
||||
the runner. Android uses the existing Docker Compose PostgreSQL 14 and Redis 7
|
||||
services on ports 5433 and 6380. GitHub-hosted macOS cannot run nested Docker
|
||||
virtualization, so iOS provisions ephemeral native PostgreSQL 14.x and Redis
|
||||
7.4.7 on those same ports and starts `pnpm --dir dev-env start:external`.
|
||||
|
||||
Each platform uploads a `nightly-e2e-<platform>-<run-id>` artifact for 14 days.
|
||||
It contains JUnit at `report.xml`, Maestro screenshots, videos, command metadata
|
||||
and `maestro.log` under `maestro/`, plus Metro, native build, mock-server, service,
|
||||
dependency, and translation logs. The workflow always uploads what was captured,
|
||||
including when setup or the native build fails before Maestro starts.
|
||||
|
||||
Add the repository secret `E2E_FAILURES_SLACK_WEBHOOK` before enabling the
|
||||
schedule. The aggregation job runs even when either platform fails and posts one
|
||||
detailed Slack notification containing both job statuses, failed flow details or
|
||||
the failed setup phase, the commit and workflow links, and links to both artifact
|
||||
sets. Successful runs do not post to Slack.
|
||||
|
||||
Before relying on the schedule, manually dispatch the workflow and verify both
|
||||
platforms against live Metro and `dev-env`, Android localhost routing, artifact
|
||||
uploads on success and failure, one Slack message for a forced failure, and no
|
||||
Slack message for an all-green run.
|
||||
|
||||
## Using Flashlight for Performance Testing
|
||||
1. Make sure Maestro is installed (optional: only for automated testing) by following the instructions above
|
||||
2. Install Flashlight by following [these instructions](https://docs.flashlight.dev/)
|
||||
|
||||
Reference in New Issue
Block a user