MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3143

(motir-core) CI: stop RUNNING apt — `install-deps` is defensive on a cache hit, and it is the only thing in the E2E path the Ubuntu mirror can hang

Cancelled
Description

Repo: motir-core (.github/actions/e2e-setup/action.yml). The THIRD attack on one flake, and the first that removes it rather than surviving it.

The two prior fixes both worked, and neither stopped it

MOTIR-1679removed the packages.microsoft.com apt sources — killed the broken-InRelease variant
MOTIR-2970added timeout 300s per attempt — turned a 6-hour job-budget hang into a ~17-minute clean failure

Each made the flake survivable. Neither made it stop, because both keep treating apt as something to get through. Observed three times on 2026-08-19 on PR #2145 alone (runs 32269024982 ×2, 32269025316, 32276806034), each identical:

Get:1 file:/etc/apt/apt-mirrors.txt Mirrorlist [144 B]
##[warning]playwright install-deps attempt N TIMED OUT after 300s; clearing the apt it orphaned
##[error]playwright install-deps failed after 3 attempts

The job dies in SETUP — no spec runs, no assertion is evaluated — and it costs 3 × 300 s per affected leg.

The way out is in the action's own comment

"the OS apt deps aren't cacheable, so we still run install-deps on a hit (fast) and the full --with-deps download only on a miss."

That cache-hit step is defensive: Chromium's OS dependencies ship on the GitHub ubuntu-24.04 runner image. And it is the only thing in the E2E path that touches apt at all. So the fix is not another retry — it is to stop running apt when nothing needs installing.

What to build

  1. PROBE, don't install. On a cache hit, try launching headless Chromium. If it launches, skip install-deps entirely — apt is never touched on the happy path, and the class disappears rather than getting a fourth mitigation.
  2. If apt DOES run and times out, verify before failing. Probe the launch again; a browser that launches is a green job with a ::warning::. A browser that does not is still a hard failure — that distinction is what keeps a genuinely missing library fatal.
  3. Keep the cache-MISS path as it is. --with-deps on a miss is the correct download and is not the failing case.

⚠️ Do NOT raise the retry count or the timeout. That is the move already made twice; it lowers probability and leaves the class.

The alternative, deliberately NOT chosen first

Running these jobs in mcr.microsoft.com/playwright:<version>-noble removes apt by construction and is the industry-standard answer. It is held in reserve because this repo has no container: jobs today, and moving there changes how the Postgres service action, the shared build artifact and the ~/.cache/ms-playwright cache are reached — a large blast radius on a path every PR depends on. Adopt it if the probe proves insufficient; the repo already mirrors images through mirror.gcr.io (.github/actions/postgres/action.yml), so the pull path is understood.

Acceptance criteria

  • The behaviour is reproduced or its absence demonstrated: a run where the cache hits and install-deps is NOT executed, shown from the job log.
  • On a cache hit with a working browser, no apt command runs. Asserted by the log, not by reading the YAML.
  • A missing OS library still fails the job — proved by a deliberately broken probe, not argued.
  • The cache-miss path is unchanged and still installs with --with-deps.
  • tests/ci-*-lane.test.ts-style guards covering this action are extended to the new arrangement, so the probe cannot be dropped silently later.
  • The retry count and the 300 s per-attempt timeout are unchanged — this card removes the need for them on the happy path, it does not tune them.
  • The flake log entry in motir-meta records the outcome so the next occurrence is measured against a fix rather than re-diagnosed.

Context refs

  • .github/actions/e2e-setup/action.yml — the cache step (~line 82), its "aren't cacheable" comment, the install-deps retry loop (~line 191)
  • .github/actions/postgres/action.yml — the existing mirror.gcr.io pull-through practice
  • MOTIR-1679 (the apt-source removal) · MOTIR-2970 (the per-attempt timeout)
  • The four failing jobs: PR #2145, runs 32269024982, 32269025316, 32276806034
  • motir-meta memory motir-core-ci-e2e-flake-log — this flake's running record