MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3569

(motir-core) 11 of ci.yml's 14 jobs declare NO `timeout-minutes`, so a hung job burns SIX HOURS and is indistinguishable from a slow one — the same gap MOTIR-2970 closed at one call site

Done
Description

Type · chore (CI configuration) · Parent · MOTIR-1464, the epic this class already lives in — it holds the direct precedent MOTIR-2970 and the sibling defect on the same job MOTIR-3497 · Discovered in · the close-out of motir run MOTIR-3563, 2026-08-26 · Repo · motir-core

The gap, measured on origin/main at c203f815

11 of the 14 jobs in .github/workflows/ci.yml declare no timeout-minutes, so each inherits GitHub's 360-minute default:

declares a ceilinginherits 6 hours
e2e (45) · e2e-at-scale (45) · deploy (30)changes · lint · structural-guards · typecheck · design-system · cli · design-guards · build · test · coverage · ci-complete

Derived by parsing the workflow rather than by reading it — the script is in the run's transcript and the counts are reproducible from the file.

The incident that surfaced it

On #2301 (run 32959226187) Vitest (2/3) ran 47.5 minutes against its siblings' 20 and 22, and against 19 minutes for the same shard on the previous run. It passed. But for 45 of those minutes nothing on the outside could tell a slow job from a hung one:

  • job logs are not downloadable while a job is in progress (BlobNotFound), so the only readable signal is a step-status poll;
  • the step list said in_progress, which is what a hang says too;
  • and with no ceiling the honest answer to "when will we know?" was "in up to six hours."

That uncertainty is the cost, and it is paid whether the job eventually passes or not.

Why the numbers say ENVIRONMENT, not content — and why that is the point

The same shard, same file set, one run apart:

fast runslow run
transform40.9 s49.6 s
import608 s292 s
setup108 s40 s
environment45 s15 s
tests3,020 s10,195 s
wall18 min45 min

Every CPU-bound phase was the same or faster; the entire delta is in test execution. The three shards ran ~403 files each and the slow one ran the fewest tests (5,646 vs 5,830 and 6,134), with zero 40P01, zero deadlocks, zero connection failures — the fewest error lines of the three.

⚠️ THE VARIANCE IS EVIDENCE FOR THIS CARD, NOT ITS DELIVERABLE, AND MUST NOT BECOME ONE. I did not root-cause it. Two readings fit different halves — degraded I/O on that VM (tests are the DB-bound phase; CPU phases were fine) or fewer vCPUs (the per-worker setup and environment costs both fell by two-thirds) — and the log carries no per-test timings, no worker count and no I/O counters to separate them. GitHub does not expose the host. A ceiling is worth having whether or not that is ever explained, which is exactly why this card asks for the ceiling and not for the explanation.

The precedent — this class already cost six hours, three times

MOTIR-2970 (done): "playwright install-deps HANGS on the Ubuntu apt mirror and burns the whole 6-hour job budget, because its 3-attempt retry has no per-attempt timeout"three legs in six hours. That fix added a timeout at one call site. The job-level gap it stood on was never closed, so the next hang anywhere else in these eleven jobs costs the same six hours again.

Observed durations, so the ceilings are measured rather than guessed

Across three recent runs (min–max, minutes): Vitest shards 19–23 (excluding the 48-minute outlier above) · build 4 · Lint + Prettier 4 · typecheck 2–3 · structural-guards 2 · design-guards 1–2 · cli 1 · design-system 1 · coverage <1 · changes <1.

45 minutes for test matches the ceiling e2e already uses and sits ~2× above its normal run, so ordinary variance — including today's 47.5-minute run — is the argument for a slightly higher number, not for no number. Pick each ceiling from the table above, not from this sentence, and state the multiple chosen.

Acceptance criteria

  1. Every job in .github/workflows/ci.yml that declares runs-on also declares timeout-minutes. The count moves from 3 of 14 to 14 of 14.
  2. Each ceiling is justified in a comment naming the job's observed duration and the headroom multiple applied, so the next person to hit one knows whether to raise it or to investigate.
  3. A ceiling is generous enough that today's 47.5-minute Vitest (2/3) would still have passed — the goal is to bound a hang, not to fail slow-but-healthy runs. Say which run was used to check this.
  4. A guard test asserts the invariant so the gap cannot silently reopen: a new job added without timeout-minutes fails the suite. tests/ci-design-guards-lane.test.ts is the shape to copy — it re-derives a list from the tree rather than trusting an enumeration.
  5. The workflow still parses and CI is green on the fix's own pull request.

Out of scope

  • The runtime variance itself. Unexplained above, deliberately. If it recurs it deserves its own card with per-test timings captured — --reporter=verbose or a slow-test threshold — which this card does not add.
  • MOTIR-3497coverage-gate-globs' own 15-second timeout, already filed. Different layer: that is a step-level timeout that fires too eagerly; this is a job-level ceiling that does not exist at all.

Context refs

  • .github/workflows/ci.ymle2e:709, e2e-at-scale:972, deploy:1197 carry timeout-minutes; test:565 and the other ten do not.
  • Run 32959226187, job Vitest (2/3) — the 47.5-minute pass; run 32956298372 is the 19-minute comparison for the same shard.
  • MOTIR-2970 — the same class at a different call site, fixed point-locally.

Resolution: open.