MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-2617

(motir-core) E2E flake, 3rd occurrence — `project-square-flow.spec.ts`'s `rank=popular` navigation stalls the full 180s on bulk-4, on the attempt AND the retry, because the retry re-uses the same degraded webServer

Done
Description

Repo: motir-core. One PR. The bulk-4 shard's heaviest spec stalls for the whole 180s test timeout on one navigation, then does it again on retry #1, and reds the PR. Three occurrences, all on innocent diffs.

The signature

[chromium] › tests/e2e/project-square-flow.spec.ts › @smoke the project square: a logged-out
visitor browses the cross-org gallery, sorts by trending, searches + filters by topic …

  Test timeout of 180000ms exceeded.
  Error: page.waitForURL: Test timeout of 180000ms exceeded.
  =========================== logs ===========================
  waiting for navigation until "load"

  > 239 |   await page.waitForURL(/[?&]rank=popular\b/);

Match on the REGEX, not the line number — it has already moved from :218 to :239 when MOTIR-2033's card(page, name) refactor landed.

Occurrences

#datePRnote
12026-07-28#1636as retry #1's rotation after the MOTIR-2033 strict-mode failure
22026-08-07#1912primary failure; three sibling sessions driving CI, jobs queued ~19 min
32026-08-10#2014primary failure; two concurrent runs + one queued

MOTIR-2033 fixed the spec's OTHER mode (an unscoped getByText racing the streamed render). This is not that one, and #1850 does not touch it.

What it is, and what it is not

Not a product regression, and each time provably so — occurrence 3's diff was two files that a logged-out /explore flow cannot import, with sibling PRs' bulk-4 green on both sides of the failing run and main green 25 minutes earlier.

It is the bulk-shard webServer-degradation class (MOTIR-1565, whose fix was scoped to sign-up 404s): bulk-4 carries the heaviest setups in the suite, the Playwright webServer creeps over a memory/CPU cliff partway through the shard, and every navigation after that point hangs. Retry #1 runs against the same dead server, so "it failed twice" is a property of the harness here rather than evidence of determinism — worth stating explicitly, because that is exactly the signal a triager reads as "real regression".

Runner starvation is a live input: both primary occurrences coincided with sibling CI runs competing for runners, occurrence 2 with ~19 minutes of queueing.

Do this

The fix is capacity and blast radius, not a spec edit and not a longer timeout — a 180s wait that needs raising is a stalled server, and raising it only makes the shard slower to go red.

  1. Find the cliff before choosing a remedy. Instrument the bulk-4 job with the webServer's RSS over the shard (or a /proc/meminfo sample per spec file) and read WHERE it crosses. Occurrence 2's tell — specs passing at 2–3s each until one hits a 180s wall — says the server degrades mid-shard rather than starting sick; confirm that before acting.
  2. Rebalance the shard. bulk-4 is a --shard=4/5 slice with no weighting, so it accumulates the heaviest multi-context setups by accident. Split by measured cost, or move the known-heavy specs (project-square-flow, project-isolation, shell-flows) onto their own leg.
  3. Fail fast instead of hanging. A dead webServer should abort the shard with a named error, not burn 6 minutes across two 180s timeouts. A health probe between spec files that reports the server unreachable turns this from "a mysterious red spec" into "the harness died", which is the whole triage cost.

Acceptance criteria

  1. The bulk-4 job records the webServer's memory over the shard, and the run's artifacts carry that series — so a recurrence names the cliff instead of needing this diagnosis again.
  2. Shard membership is derived from measured per-spec cost rather than Playwright's unweighted --shard=N/5 slice, OR the heavy specs named above are moved to a dedicated leg. Whichever is chosen, a test asserts the mapping so a future spec cannot silently rejoin the overloaded shard.
  3. A degraded webServer aborts the shard with an error naming it, within one spec's timeout rather than after two — asserted against a deliberately-killed server, so the check is falsifiable.
  4. tests/e2e/project-square-flow.spec.ts is not edited, and no waitForURL / test timeout is raised. If the spec needs a change, that is a different bug with different evidence.
  5. The three prior occurrences are named in the fix's PR body with their run ids, so the next triager can confirm the signature matches rather than re-deriving it.

Out of scope

MOTIR-2033's strict-mode locator mode (fixed, #1850) and MOTIR-1742's docker pull service-container mode (fixed, #1635). Both are different failures on adjacent surfaces; do not fold them in.