MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3484

(motir-core) COLLAPSE the INDEX supervisor — `indexFleetSteps.ts` becomes one `while` loop over the service's own composition, and the `maxDuration` comments are CORRECTED rather than deleted

Done
Description

Collapse the index fleet's stepped supervision into ONE composition — an ordinary loop with real awaits — keeping exactly the durable boundaries the restart decision names, and correcting the comments that explain the shape rather than deleting them.

What is there today, and why there are TWO of it

lib/jobs/indexFleetSteps.ts drives codeGraphIndexDispatchService as durable steps: resolve-targetassert-fleet-configured → per project index-admit:<pid>:<n> (up to INDEX_ADMISSION_BUDGETS.maxAttempts = 60, each its own step id) → index-boot:<pid>index-wait:<pid>:<n> / index-poll:<pid>:<n> (up to INDEX_FLEET_TIME_BUDGETS.maxPollIterations = 500) → index-settle:<pid>cancel-offboarding.

The same service already carries the collapsed form. codeGraphIndexDispatchService.runIndexContainer composes admit → boot → poll-loop → settle with a plain sleep, and waitForAdmission is admitWithBackoff without the step ids. Its own doc comment says why it is not used:

"⚠️ NOT THE PRODUCTION PATH. The job (MOTIR-2027) drives … as separate durable steps; calling this from a job would rebuild the hour-long invocation MOTIR-2007 removed for CI."

That sentence is a statement about app/api/inngest/route.ts's maxDuration = 300. Dockerfile ends CMD ["node", "server.js"] and the worker is its own Fly process group with a renewed lease — so the reason the two compositions exist has gone, and the card's real deliverable is that there is ONE of them again.

The change

  1. Parameterise the service's composition by an OPTIONAL step API rather than keeping two copies. runIndexContainer / waitForAdmission take the same { run, sleep } seam the job can pass ctx.step into; a caller that passes nothing gets today's in-process behaviour, which is what the suite and any script already rely on. One composition, two callers — instead of two compositions that must be kept in agreement by hand.
  2. Apply the restart decision to the seam. The operations that PROVISION, CLAIM or TEAR DOWN stay memoized step.runs; the interval becomes a plain await and the poll an ordinary call. Do not invent the line here — read it off the amendment that card lands.
  3. indexFleetSteps.ts shrinks to what is genuinely its own: resolve-target, the config gate, the per-project fan-out, the dispatchId hoist, cancel-offboarding, and the ledger contract. admitWithBackoff and the stepped poll loop go, because the service now owns both.
  4. CORRECT the comments; do not delete them. Four blocks in this file argue from maxDuration: "⚠️ WHY STEPPED, AND NOT A LOOP INSIDE ONE STEP", "⚠️ SO WHAT THIS LOOP OCCUPIES IS ~128 SUB-SECOND STEPS", "⚠️ STEP IDS ARE KEYED BY projectId, NEVER BY LOOP POSITION", and "⚠️ THE ADMISSION CAP IS A LOOP OF STEPS". Each must end up saying that the constraint it reasons from was Vercel's and is gone, and what the shape is now — a future reader must be able to see that the world changed, not merely that the code did. The same applies to runIndexContainer's "NOT THE PRODUCTION PATH" comment, which becomes false the moment this card lands.

Scope boundary

ENDS at: one composition, driven by the job through the step seam, with pnpm test green.

Does NOT change any observable behaviour. The poll cadence (indexPollWaitMs), the admission backoff (indexAdmissionWaitMs), every budget in INDEX_FLEET_TIME_BUDGETS and INDEX_ADMISSION_BUDGETS, the exit classification and the failure messages are all unchanged. A criterion asserting otherwise is on the wrong card.

Does NOT touch codeGraphIndexAdmissionService or lib/ciFleet/limits.tsMOTIR-3417 forbids it, because container admission is a different resource and a regression there costs money.

Does NOT touch the CI runnerlib/jobs/definitions/ciRunnerFleet.ts and lib/services/ciRunnerBootService.ts belong to the CI collapse, which is a sibling and not a dependency: the two share no file.

Does NOT edit lib/jobs/definitions/codeGraphIndex.ts or codeGraphRefresh.ts beyond the comment corrections their own headers need — they call runIndexFleetSteps and keep calling it.

HANDS OFF the DOCUMENTS. docs/jobs.md rule 1 ("A STEP, NOT A RUN, IS THE UNIT THE PLATFORM'S TIMEOUT APPLIES TO") and docs/decisions/code-graph-index-fleet.md §2 / §11 make the same claim this card falsifies in code, and they belong to the docs card. Correct the CODE comments here; leave the documents alone.

Acceptance criteria

  • codeGraphIndexDispatchService exposes ONE supervision composition, and git grep -n "for (let iteration" lib/services/codeGraphIndexDispatchService.ts lib/jobs/indexFleetSteps.ts finds it in exactly one place.
  • runIndexFleetSteps no longer contains a poll loop or an admission-backoff loop, and admitWithBackoff is gone from the file.
  • The number of job_step rows a completed index writes is a small CONSTANT, independent of how many times it polled — asserted by a test that drives a supervised run at millisecond budgets and counts the rows. (This is the property the collapse buys: today each poll writes a sleep checkpoint and a result row, and the handler replays every earlier one on each resume.)
  • The LEDGER CONTRACT is unchanged: ONE job_run per repo, succeeded, ONE output.repoRef; a run that did not index every project still throws IndexDispatchFailedError carrying the dispatch service's named exit class, not a bare number.
  • The three no-op resolve-target verdicts still return before the config gate, so a vanished tenant is still a clean no-op whose reason is the run's ledger output.
  • The step ids that REMAIN are still keyed by projectId and never by loop position, and the file still says why.
  • INDEX_FLEET_TIME_BUDGETS and INDEX_ADMISSION_BUDGETS are byte-identical, asserted against the shipped values rather than by inspection.
  • git diff --stat shows no change under lib/services/codeGraphIndexAdmissionService.ts or lib/ciFleet/.
  • Every comment block naming maxDuration in the files this card touches either states that the ceiling no longer applies, or is gone along with the code it described — verified by re-reading git grep -n maxDuration over the changed files and dispositioning each hit.

Context refs

  • lib/jobs/indexFleetSteps.ts — the stepped driver, its four maxDuration comment blocks, and the ledger contract to preserve
  • lib/services/codeGraphIndexDispatchService.tsrunIndexContainer, waitForAdmission, pollIndexContainer, settleIndexContainer, the budgets, and the "NOT THE PRODUCTION PATH" comment that stops being true
  • lib/jobs/definitions/codeGraphIndex.ts · lib/jobs/definitions/codeGraphRefresh.ts — the two callers, unchanged in behaviour
  • lib/jobs/engine/step.ts — the JobStepApi shape the seam is modelled on
  • Dockerfile · fly.toml — the long-lived process the comments must now describe
  • MOTIR-2027 — the card that built the stepped shape, and whose premise this reverses
  • MOTIR-2384 — the move off Vercel that made the premise false