(motir-core) 19 of the 30 most expensive test files landed on ONE Vitest leg — `--shard` slices the alphabet, so the `tests/integration/**` cluster travels together

Type · chore (CI configuration) · Parent · MOTIR-1464 · Repo · motir-core · Measured · run 33251966134 (PR #2453), the first 8-shard run · Follows · MOTIR-3902, which predicted this and scoped it out

The gap, measured on the first 8-shard run

MOTIR-3902 took the Vitest matrix 3 → 8 legs and said the spread would widen. It did, and by more than "widen" suggests. Per-leg in-file test time, summed from all eight job logs:

2.52x spread. The file COUNTS are even to within one — it is the cost that is not.

The mechanism, and why it is not bad luck. --shard=i/n keeps whole files together in DISCOVERY (alphabetical) order and slices contiguously, so nothing in the split knows what a file costs. The expensive files cluster by DIRECTORY, and a directory is contiguous in that order. 19 of the 30 most expensive files in the suite landed on leg 6 — it inherited most of tests/integration/**, including plansService.test.ts (125.8 s), abandonedPlanSweep.test.ts (83.3 s), autoPlanCadence.test.ts (79.0 s), statusDerivation.test.ts (74.0 s), work-items/service.test.ts (71.1 s) and parentStatusRollup.test.ts (68.2 s). This recurs every run until the split changes; it is not a flake.

What it costs, stated honestly — it is TWO minutes, not five. Leg 6 is the critical path by 43 seconds and no more:

12:28:54  Playwright E2E (bulk-3) ends      ← the E2E path is done here
12:29:37  Vitest (6/8) ends
12:29:40  Vitest coverage starts
12:31:03  Vitest coverage ends
12:31:07  CI complete                        → 17.7 min

Balance the legs and Vitest + coverage lands ~12:26, at which point E2E becomes binding at ~15.6 min. So this card is worth ~2 min of wall clock now, and is a PREREQUISITE for the E2E half being worth anything: neither lane alone takes CI below ~15.5 min. See MOTIR-3903 for the other half.

What a cost-based split is worth. LPT bin-packing the same 1364 measured costs packs to a 1.00x spread — every leg 1458 s of test time, 43% off leg 6's long pole. The single most expensive file (125.8 s) is the floor no packing beats, and it is far below the 1458 s target, so there is no lumpiness obstacle.

The design, and where it must NOT copy the E2E plan

tests/e2e/shard-plan.ts (MOTIR-2617) is the pattern for the packer, and its guard asserts every spec has a measured cost entry or the build fails. ⚠️ That guard is correct for ~80 specs added a few a month and WRONG for 1364 unit-test files added continuously — copied over, it fails every PR that adds a test until someone hand-measures it. Do not copy it.

Invert the dependency instead:

1. The leg membership is computed from the file list DISCOVERED on disk, not from the cost table's keys — glob tests/**/*.test.{ts,tsx} minus STRUCTURAL_GUARD_SPECS, exactly as vitest.config.ts resolves it. Totality is then structural: every file that exists is packed onto exactly one leg, whether or not anyone measured it.
2. Cost is a LOOKUP WITH A DEFAULT — FILE_COST_SECONDS[file] ?? DEFAULT_COST_SECONDS. A new test file gets the default, lands on a leg, and runs. It costs a little balance, never a red build and never a skipped test.
3. The guard asserts the properties that matter: every discovered file on exactly one leg, no file twice, the assignment identical when computed independently (all eight legs compute it separately — a non-deterministic packer silently drops or double-runs files), balance within tolerance, and the ci.yml matrix legs matching the plan's leg ids.

⚠️ Two traps that are silent if missed

• The blob filename collides. vitest's blob reporter resolves outputFile = this.options.outputFile ?? getOutputFile(config, 'blob'), and only then falls back to `.vitest-reports/blob-${shard.index}-${shard.count}.json` when --shard is set, else .vitest-reports/blob.json. Dropping --shard therefore makes all eight legs write the SAME filename, and the coverage job downloads them with merge-multiple: true — which flattens them into one directory, so seven blobs overwrite one another and the ≥90% gate measures a single leg. Pass --outputFile.blob=.vitest-reports/blob-<leg>.json explicitly.
• The guard must live in the STRUCTURAL-GUARD lane, not the sharded suite. A guard that rides on a leg is a guard the plan can assign away — and the plan is what it exists to check. Add it to STRUCTURAL_GUARD_SPECS (which vitest.config.ts reads as its exclude, so the two cannot drift).

The cost model, and what it does NOT know

Per-file cost = measured in-file test seconds + a flat 1.92 s. The constant is the suite's non-test per-file overhead — (import 2259.6 s + transform 224.0 s + environment 133.7 s) / 1364 files across the eight legs.

Say plainly what this misses: import is not uniform per file — it ran 0.87 s/file on leg 2 and 2.01 s/file on leg 5, because a file's cost depends on which modules it pulls in, and the log reports import only per LEG. A flat constant over-charges the light legs and under-charges the heavy ones. It is a first approximation, it is much better than counting files, and it is stated here so the next reader does not mistake it for a per-file measurement. Re-measure from the first green run that uses the plan — that run reports each leg's real phases against a known membership, which is the reading this model cannot produce for itself.

Acceptance criteria

1. Leg membership comes from a bin-packer over measured costs, and every discovered test file is assigned to exactly ONE leg. Tests N passed summed across the eight legs equals what run 33251966134 reported on its SHA (19 773, adjusted for files the diff adds or removes).
2. The measured leg spread on this PR's own run is below 1.3x (from 2.52x), and no leg exceeds ~11 min. Quote the eight readings in the PR body.
3. A new test file with no cost entry is assigned a leg and RUNS. Prove it in the guard, not by assertion in prose — a test that adds a synthetic unmeasured filename to the discovered set and asserts it lands on exactly one leg.
4. The coverage merge job still gates the merged report at the same ≥90% per-file thresholds, with eight distinct blob files reaching it. A leg's blob must not overwrite another's.
5. The guard runs in the structural-guard lane and fails on: a file assigned to no leg, a file assigned twice, a non-deterministic assignment, and a ci.yml matrix that does not match the plan's leg ids.
6. tests/ci-job-timeouts.test.ts stays green.

Out of scope

• The leg COUNT. It stays at 8. With the packer in place the count becomes a one-line dial, and turning it is a separate decision that wants a measurement of the balanced lane first — which this card produces and does not have.
• --no-isolate and Postgres durability — still the MOTIR-3902 list, still unmeasured at scale.

Context refs

• tests/e2e/shard-plan.ts + tests/e2e-shard-plan.test.ts — the packer to copy and the guard NOT to copy wholesale.
• .github/workflows/ci.yml — the test matrix and its vitest run … --shard step; the coverage job's merge-multiple download.
• vitest.collect.config.ts — the shards' config, where the leg's include is resolved.
• tests/helpers/structuralGuardLane.ts — STRUCTURAL_GUARD_SPECS, the lane the new guard joins.

Resolution: open.