Add nightly Maestro E2E workflow

This commit is contained in:
Samuel Newman
2026-07-17 13:04:12 +03:00
parent 21d53ce0f8
commit 90267eb655
11 changed files with 841 additions and 1 deletions
+38
View File
@@ -29,6 +29,44 @@ adb reverse tcp:3000 tcp:3000
- In a second tab, run `pnpm e2e:build`
- In a third tab, run `pnpm e2e:run __e2e__`
## Nightly Maestro CI
The `Nightly Maestro E2E` GitHub Actions workflow runs every day at 04:00 UTC
and can also be started from the Actions tab with **Run workflow**. It runs iOS
and Android concurrently, but each platform runs all of `__e2e__/config.yml`
sequentially on one explicitly selected simulator or emulator. The flows share a
stateful mock-server manager, so the suite must not be sharded.
The jobs run Maestro CLI 2.6.1 locally on GitHub Actions; Maestro Cloud is not
used. iOS runs on `macos-26-xlarge` with Xcode 26.4. Android runs on
`Linux-x64-32core`. Both use Java 17 and the Node and pnpm versions declared in
`package.json`.
The mock-server manager listens on host port 1986 and creates test services on
port 3000. Metro listens on 8081. Android reverses ports 3000 and 8081 into the
emulator; port 1986 remains host-side because Maestro JavaScript calls it from
the runner. Android uses the existing Docker Compose PostgreSQL 14 and Redis 7
services on ports 5433 and 6380. GitHub-hosted macOS cannot run nested Docker
virtualization, so iOS provisions ephemeral native PostgreSQL 14.x and Redis
7.4.7 on those same ports and starts `pnpm --dir dev-env start:external`.
Each platform uploads a `nightly-e2e-<platform>-<run-id>` artifact for 14 days.
It contains JUnit at `report.xml`, Maestro screenshots, videos, command metadata
and `maestro.log` under `maestro/`, plus Metro, native build, mock-server, service,
dependency, and translation logs. The workflow always uploads what was captured,
including when setup or the native build fails before Maestro starts.
Add the repository secret `E2E_FAILURES_SLACK_WEBHOOK` before enabling the
schedule. The aggregation job runs even when either platform fails and posts one
detailed Slack notification containing both job statuses, failed flow details or
the failed setup phase, the commit and workflow links, and links to both artifact
sets. Successful runs do not post to Slack.
Before relying on the schedule, manually dispatch the workflow and verify both
platforms against live Metro and `dev-env`, Android localhost routing, artifact
uploads on success and failure, one Slack message for a forced failure, and no
Slack message for an all-green run.
## Using Flashlight for Performance Testing
1. Make sure Maestro is installed (optional: only for automated testing) by following the instructions above
2. Install Flashlight by following [these instructions](https://docs.flashlight.dev/)