From 020a85f0294cc9519125c599c3935dc02cded959 Mon Sep 17 00:00:00 2001 From: Austin McKinley Date: Sun, 26 Jul 2026 16:30:57 -0700 Subject: [PATCH] Publish one bundle version to both OTA origins, not two bundleUpdate.sh and denisPublish.sh each called `date +%s`, so a single dual-write gave the same bytes two different bundle versions -- observed 1785102575 (denis) and 1785102614 (ota1), 39 seconds apart, for commit b662cd43b. That is not cosmetic. The bundle version is a path segment in every asset URL the manifest hands the client: https://updates.bsky.app/file/1.130.0//bundles/ Both origins mint those URLs against updates.bsky.app, which resolves to whichever origin Bunny currently points at. So each origin was serving a manifest whose assets only IT has. A manifest fetched from one origin and assets fetched from the other 404s -- which is exactly what a rollback of the Bunny origin does if it lands between a client's manifest fetch and its asset fetch. The dual-write existed to make rollback safe and was quietly making it unsafe. Pin the version once in the job and let both scripts inherit it. Both fall back to `date +%s` when BUNDLE_VERSION is unset, so the single-publisher callers are unchanged: pull-request-commit.yml (denis only) and `pnpm make-deploy-bundle`. The fallback uses `:-` rather than `-`, so an empty value also falls back -- `denis publish` rejects a non-numeric bundle version, so an empty one must never propagate. This does not make the two manifests byte-identical. createdAt still differs (21:49:35Z vs 21:50:18Z for the commit above) because ota1's legacy uploader stamps it server-side on receipt; the client posts a tarball, not a timestamp, so nothing here can align it. That residual is inert in a way the bundle version was not: the manifest id is content-addressed on metadata.json and is identical across origins, so manifestHandler's `currentUpdateID == entry.Manifest.ID` check makes a client that switches origins see the same update rather than a newer one. It goes away with the dual-write in Phase 5. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/bundle-deploy-eas-update.yml | 12 ++++++++++++ scripts/bundleUpdate.sh | 6 +++++- scripts/denisPublish.sh | 11 ++++++++++- 3 files changed, 27 insertions(+), 2 deletions(-) diff --git a/.github/workflows/bundle-deploy-eas-update.yml b/.github/workflows/bundle-deploy-eas-update.yml index 28308e5073..21393c9973 100644 --- a/.github/workflows/bundle-deploy-eas-update.yml +++ b/.github/workflows/bundle-deploy-eas-update.yml @@ -204,6 +204,18 @@ jobs: SENTRY_DIST=${{ steps.env.outputs.bundle-identifier }} pnpm export + # Pin ONE bundle version for both publishes below. Each script used to call + # `date +%s` itself, so the same bytes reached denis and ota1 under versions + # seconds apart (observed: 1785102575 vs 1785102614). The version is part of + # the asset URL path, so each origin then served a manifest referencing a + # path only it had -- meaning a manifest fetched from one origin and assets + # fetched from the other 404. Both scripts fall back to `date +%s` when this + # is unset, so single-publisher callers are unaffected. + - name: 🔢 Pin bundle version + if: ${{ !steps.fingerprint.outputs.includes-changes && + !steps.version.outputs.version-changed }} + run: echo "BUNDLE_VERSION=$(date +%s)" >> "$GITHUB_ENV" + # denis on EKS has been the sole origin for updates.bsky.app since # 2026-07-26, so it publishes FIRST: it is the path that actually serves # clients. The legacy ota1 upload runs after it, and exists only so that diff --git a/scripts/bundleUpdate.sh b/scripts/bundleUpdate.sh index 0685ecd108..35bed14ed0 100644 --- a/scripts/bundleUpdate.sh +++ b/scripts/bundleUpdate.sh @@ -14,7 +14,11 @@ if [ -z "$RUNTIME_VERSION" ]; then fi cd bundleTempDir || exit -BUNDLE_VERSION=$(date +%s) + +# Shared with denisPublish.sh when both run in one job -- see the note there. +# Both origins must receive the same bundle version for the same bytes, because +# the version is part of the asset URL path. +BUNDLE_VERSION="${BUNDLE_VERSION:-$(date +%s)}" # This MUST address ota1's own origin hostname, never updates.bsky.app. # diff --git a/scripts/denisPublish.sh b/scripts/denisPublish.sh index f983bf21d1..c6a30b77b9 100755 --- a/scripts/denisPublish.sh +++ b/scripts/denisPublish.sh @@ -19,7 +19,16 @@ if [ -z "$RUNTIME_VERSION" ]; then RUNTIME_VERSION=$(cat package.json | jq '.version' -r) fi -BUNDLE_VERSION=$(date +%s) +# Accept a caller-supplied bundle version so that a dual-write publishes the SAME +# version to every origin. When this script and bundleUpdate.sh each called +# `date +%s` independently they produced versions seconds apart for identical +# bytes -- observed 1785102575 (denis) vs 1785102614 (ota1) for one commit. Since +# the version is part of the asset URL path, the two origins then served +# manifests pointing at paths only one of them had, so the manifest and its +# assets had to come from the same origin or the fetch 404s. Falling back to +# `date +%s` keeps standalone callers (PR previews, `pnpm make-deploy-bundle`) +# working unchanged. +BUNDLE_VERSION="${BUNDLE_VERSION:-$(date +%s)}" DENIS_CDN_DOMAIN="${DENIS_CDN_DOMAIN:-updates.bsky.app}" DENIS_S3_BUCKET="${DENIS_S3_BUCKET:-bsky-denis-ota-prod}"