MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3417

Cut the container supervisors over — and COLLAPSE the stepped shape now that nothing times out

Done
Description

Moves the container supervisors onto the engine — and then does the thing that makes this epic worth more than its cost: deletes the stepped shape they only ever had because of Vercel.

The supervisors

system.code-graph-index, system.code-graph-refresh (which carries the substrate's only debounce — a 2 m period, 15 m timeout, keyed on installation + owner + repo) and system.ci-runner-boot. All three drive lib/jobs/indexFleetSteps.ts or its CI sibling: boot a container, poll it to completion, tear it down.

The collapse, and why it is legitimate now

indexFleetSteps.ts says in its own words why it is stepped:

"An index run is minutes; app/api/inngest/route.ts pins maxDuration = 300 ... A STEP, NOT A RUN, IS THE UNIT THE PLATFORM'S TIMEOUT APPLIES TO, so the WAITING is ctx.step.sleep."

That ceiling is Vercel's. Dockerfile ends CMD ["node", "server.js"] and motir-core has run as a long-lived Fly process since MOTIR-2384. A supervisor can now be an ordinary async function with a while loop and an await. What that removes:

  • step ids keyed by projectId and by loop iteration, and the rule explaining why they must never be positional;
  • admitWithBackoff's retry loop expressed as N separately-memoized steps purely so Inngest would not freeze the first deferred answer;
  • the stepped finally — teardown written as a step reachable on every path out of the loop, because a real finally could not be trusted across invocations;
  • roughly 128 step.run round trips per 30-minute index, each one a database write.

The durability property must NOT be lost with the ceremony. A supervisor that dies mid-index — a deploy, a crash — still has to resume rather than orphan a running container. That is what the engine's durable step.sleep is for, and this story decides per loop which waits stay durable steps and which become plain awaits. A deploy is a routine event, not an exception, so "it works until we deploy" is not an acceptable outcome.

Scope boundary

ENDS at: all three supervisors on the Postgres engine, the debounce reproduced, the stepped scaffolding removed where the invocation ceiling was its only justification.

HANDS OFF: deletion of the Inngest SDK to the retirement story. Does NOT change container admission, the Fly spend ceiling in codeGraphIndexAdmissionService / lib/ciFleet/limits.ts, or any index's own behaviour.

Verification recipe

Push to a connected repo's default branch. Watch a refresh coalesce over its debounce window, boot a container, poll it, and settle — one job_run row per repo, succeeded, with its output.repoRef. Deploy the app mid-index and confirm the supervisor resumes rather than orphaning the container.

Acceptance criteria

  • All three supervisors run on the Postgres engine; a repo push still produces exactly ONE coalesced refresh run carrying the latest event.
  • A supervisor survives a worker restart mid-run and resumes supervising its container — asserted by a test that actually restarts, not by reading the code.
  • The stepped scaffolding whose only justification was maxDuration is removed, and the comments that explained it are corrected rather than deleted — a future reader must be able to see that the constraint went away, not merely that the code changed.
  • The ledger contract in indexFleetSteps.ts is unchanged: ONE job_run per repo, succeeded, one output.repoRef, and a run that did not index every project still throws.
  • codeGraphIndexAdmissionService and lib/ciFleet/limits.ts are untouched — container admission is a different resource and a regression there costs money.

Context refs

  • lib/jobs/indexFleetSteps.ts — the supervision loop and the comments to correct
  • lib/jobs/definitions/codeGraphRefresh.ts — the debounce, and the no-concurrency block already corrected by MOTIR-3246
  • lib/services/codeGraphIndexDispatchService.tsINDEX_FLEET_TIME_BUDGETS, the poll budgets
  • tests/jobs/debounce-burst.test.ts — the existing coalescing guard