(motir-core) With Vitest balanced, E2E becomes CI's ceiling at ~15.5 min — five bulk legs carry 1705 s of test work behind a 178 s per-leg boot nobody has costed
Type · chore (CI configuration) · Parent · MOTIR-1464 · Repo · motir-core · Measured · run 33251966134 (PR #2453) · Pairs with · MOTIR-3912 — neither card alone takes CI below ~15.5 min
Why this is worth doing NOW and was not worth doing last week
Until MOTIR-3902 the Vitest lane ran 24.8 min and E2E's 11.3 was invisible behind it. On run 33251966134 the two lanes are 43 seconds apart:
12:28:54 Playwright E2E (bulk-3) ends
12:29:37 Vitest (6/8) ends
12:31:07 CI complete → 17.7 min
MOTIR-3912 balances Vitest to ~10 min per leg, which lands Vitest + coverage around 12:26 and makes E2E the binding constraint at ~15.6 min (changes 0.5 + build 4.4 + bulk-3 10.9). This card moves that constraint. Doing it WITHOUT MOTIR-3912 buys nothing at all — Vitest's leg 6 would still end last.
Where an E2E leg's 10.9 minutes actually goes
Read off bulk-3's job log, which is the only place the split is visible:
| phase | time |
|---|---|
checkout + Postgres + e2e-setup (job steps) | 83 s |
inside the Run E2E step, before the first test | 178 s |
test execution (Running 99 tests using 1 worker 12:22:21 → 99 passed 12:28:50) | 389 s |
| job total | 650 s (10.9 min) |
The 178 s is the dual-next start webServer boot plus seeding, and it is re-paid by every leg. Together with the job steps that is 261 s of fixed cost per leg against 389 s of work — the reason ci.yml's own comment stopped at five legs and said "more legs past this give diminishing returns for the cost". That comment is right about the SHAPE and has never been given the number; this card supplies it.
Total test work across the five legs, netting out the boot: (459−178) + (522−178) + (567−178) + (509−178) + (538−178) = 1705 s.
| legs | test/leg | step | job | fixed cost total |
|---|---|---|---|---|
| 5 (today) | 341 s | 519 s | 602 s avg (650 s worst) | 1305 s |
| 8 | 213 s | 391 s | 474 s (7.9 min) | 2088 s |
| 10 | 170 s | 348 s | 431 s (7.2 min) | 2610 s |
8 is the knee. Going 5 → 8 takes ~3 min off the slowest leg for ~13 extra runner-minutes; 8 → 10 buys 0.7 min more for another 9. Below ~8 legs the boot dominates and the curve flattens — which is what "diminishing returns" meant, now with a number attached.
E2E path after this card: changes 0.5 + build 4.4 + leg 7.9 ≈ 12.3 min, against a balanced Vitest path of ~11.9. The two lanes land level, which is the right place to stop.
The recorded costs have DRIFTED and must be re-measured in the same change
SPEC_COST_SECONDS in tests/e2e/shard-plan.ts is sourced from two green runs of 2026-08-10 plus a handful of locally-measured additions. tests/e2e-shard-plan.test.ts asserts the legs balance to within 15% of those recorded numbers — which stays green while reality drifts, because the guard measures the table against itself.
It has drifted. Net of the 178 s boot, the five legs ran 281 / 344 / 389 / 331 / 360 s — a 38% spread, not 15. Re-packing stale costs across 8 legs would carry that error into a finer split, where it hurts more. So: re-measure from the playwright-report-bulk-* artifacts of run 33251966134 (all five present, unexpired) by summing result.duration per spec file, and update the table in the same PR. Keep the file's provenance discipline — say which runs each number came from, and keep the local-measurement warnings intact for entries this run does not cover.
Acceptance criteria
BULK_LEG_IDSis eight legs;ci.yml'se2ematrix lists the same eight, andtests/e2e-shard-plan.test.ts's ci.yml cross-check passes against them.SPEC_COST_SECONDSis re-measured from run33251966134's fiveplaywright-report-bulk-*artifacts, with the provenance note updated to name that run and date. Entries that run does not cover keep their existing value AND their existing provenance warning — do not silently re-stamp a number this measurement did not take.- On this PR's own run the eight legs' test-execution time (step time minus each leg's own pre-test window, read from its
Running N testsline) spreads under 1.25x, and no bulk leg's job exceeds ~9 min. Quote the eight readings. - Every spec still runs exactly once: summed
N passedacross the eight legs equals what the five legs reported on run33251966134, adjusted for specs the diff adds or removes. The existing "assigns every spec to exactly one leg" guard covers the mechanism; the count is the receipt. tests/ci-job-timeouts.test.tsstays green, ande2e'stimeout-minutes: 45is revisited against the new per-leg readings — same rule as MOTIR-3569: a ceiling bounds a hang, never a slow-but-healthy run, so it may stay high, but its comment must not keep quoting a five-leg membership.
Out of scope — and one of them is the bigger prize
- ⚠️ The 178 s pre-test boot itself. It is now 45% of a leg and, at eight legs, 2088 s of runner time per run — more than the test work it protects. Halving it would beat this card. It is out of scope because it is a different investigation (dual
next startvs. seeding vs.migrate deploy, none of them separated yet by measurement) and because sizing the legs first is what makes the boot's share legible. File it as its own card once this lands — do not fold it in. - The
buildjob (4.4 min), which every leg waits on. e2e-at-scaleand the@a11ylegs — the a11y pair already runs 5.6–5.9 min and is not near the ceiling.
Context refs
tests/e2e/shard-plan.ts—BULK_LEG_IDS,SPEC_COST_SECONDS,assignBulkLegs; the packer is already total and needs no change beyond the leg list.tests/e2e-shard-plan.test.ts— the guard, including its ci.yml matrix cross-check and the 15%-of-recorded-cost balance assertion.playwright.config.ts— consumesE2E_SHARDvialegTestMatch..github/workflows/ci.yml— thee2ematrix (fivebulk-*entries sharing one--grep-invert).- Run
33251966134— the measurement, and itsplaywright-report-bulk-{1..5}artifacts.
Resolution: open.
Comments (0)
No comments yet — be the first to weigh in.