(motir-core) Decision — what survives a WORKER RESTART mid-supervision: which waits stay durable steps, which become plain `await`s

Settle, and RECORD, what a supervisor is allowed to lose when the worker process dies mid-supervision — because that answer is what decides which of today's ctx.step calls survive the collapse and which become ordinary awaits.

The question

MOTIR-3417 says the stepped shape exists only because of Vercel's maxDuration, and that the durability property must not be lost with the ceremony. Both halves are true, and they are not the same property, which is why this cannot be decided card-by-card while writing the loops. State the rule once and let the two collapse cards apply it.

Precisely: for each ctx.step.run / ctx.step.sleep in the two supervision loops, does a worker restart at that point have to resume where it left off, or is re-entering the handler from the top acceptable?

What is already true, verified on origin/main@7e97e2ed

• A long run is the engine's NORMAL case, not an edge case. lib/jobs/engine/worker.ts says so in its own header: a claim carries a lease (LEASE_MS 60 s, renewed every RENEW_MS 20 s), "a run legitimately longer than the lease is the normal case, not the exception (the container supervisors sleep for half an hour)", and a reclaim REFUNDS the attempt. So an in-memory wait does not need step.sleep to survive a live worker — the heartbeat covers that.
• A reclaim re-invokes the handler FROM THE TOP. lib/jobs/engine/runner.ts's runQueuedJob builds a fresh context and calls def.handler; lib/jobs/engine/step.ts then serves each step.run from job_step if a row exists. So a memoized step is what makes re-entry cheap and safe, and an un-memoized side effect is what makes it dangerous.
• The cross-pass identities survive. buildEngineContext sets event.id from run.eventId, which is stable for the life of the run — so indexFleetSteps.ts's dispatchId = ctx.event.id ?? ctx.runId, the value that owns the admission slot (MOTIR-2160), holds across passes on this engine as it did on Inngest.
• The wall clock is anchored to the SESSION, not to the loop. pollIndexContainer computes elapsed from session.bootedAt — a field on the memoized boot result — so indexTimeoutMs (1_800_000) keeps bounding a container across a restart even if the loop counter resets. The same is true of the CI fleet's jobTimeoutMs (3_600_000).
• startedAt re-heals on the happy path and NOT on the failed-read path. pollIndexContainer re-derives startedAt from the provider's own status (if (status.startedAt && !startedAt) …) before consulting deadlineVerdict, so a reset in-memory startedAt is repaired by the first successful read. In the catch arm it is not: deadlineVerdict(null) is evaluated with the reset value, so a reclaim landing on an unreadable provider can read elapsed >= bootDeadlineMs (120_000) for a container that has in fact been running for much longer. This is the one place the loop's in-memory state is load-bearing, and the decision owes it an explicit disposition.

The options to weigh, and the evidence each is judged on

1. Everything stays a durable step (today's shape, ported). Correct, and it is what the engine currently gives for free — but on our own engine each step.sleep is a re-enqueue, a re-claim and a replay of every earlier step, so a loop that polls N times performs on the order of N² memo lookups. On Inngest the waits were genuinely free; here they are not.
2. Nothing is durable — a plain in-process loop with no memoized steps at all. Rejected on its face, and say why in the record: a restart would re-execute the boot, which provisions a SECOND billed container and takes a second admission slot.
3. Durable at the SIDE-EFFECT boundaries, plain await for the WAITING — memoize the operations that provision, claim or tear down; make the interval and the poll ordinary calls. A restart then replays the boot from its memo, re-attaches to the same session, and carries on watching.

The recommendation this card is expected to reach is (3), and it is not a foregone conclusion — the card's job is to state the rule crisply enough that a reader can apply it to a step it does not name, and to answer the three consequences (3) has:

• What the poll ITERATION counter means after a reclaim — it restarts, and the ceilings (MAX_POLL_ITERATIONS = 500 for the index fleet, 2_000 for CI) are therefore no longer a bound on total polls per container. Say whether that is acceptable given that bootedAt-anchored wall clock is the real bound, or require the count to be derived from elapsed time instead.
• The catch-arm startedAt hazard above — either persist the observed startedAt on the session, or re-order the failed-read arm so a boot-deadline verdict cannot be reached from a reset value, or accept it with the reason written down.
• Whether a reclaim may re-ask ADMISSION. With the backoff loop collapsed, a restart re-enters admitIndexContainer for a dispatch that may already hold a slot. IndexAdmissionVerdict has an already_held outcome and the slot is keyed by dispatchId, so this looks safe — the record must say so having checked codeGraphIndexAdmissionService, not having assumed it.

Scope boundary

ENDS at: an amendment to docs/decisions/job-queue-foundation.md stating the rule, its three consequences, and the reasoning — plus the same rule restated in one paragraph in the step.ts header, where the next person writing a loop will actually be standing.

Changes NO supervision code. The two collapse cards apply this; this card decides it.

Does NOT re-open whether the supervisors move to the engine at all (that is the story), whether a debounce is needed (that is its own sibling), or the admission cap and lib/ciFleet/limits.ts (the story forbids touching either).

Acceptance criteria

• docs/decisions/job-queue-foundation.md carries a new amendment stating, as a rule a reader can apply to a step it does not name, which ctx.step calls a supervision loop must keep and which it may drop.
• The rule is justified against the worker's actual lease and reclaim behaviour, citing lib/jobs/engine/worker.ts and lib/jobs/engine/step.ts rather than restating this card.
• Each of the three consequences above — the iteration counter, the catch-arm startedAt read, the re-asked admission — has an explicit disposition in the record, and any that is deferred rather than settled names the work item that owns it (a deferral naming no key is not a disposition).
• The already_held claim is discharged by reading codeGraphIndexAdmissionService, with the function and the outcome named.
• lib/jobs/engine/step.ts's header carries the one-paragraph restatement, so the rule is where a loop author stands.
• No file under lib/jobs/definitions/, lib/jobs/indexFleetSteps.ts, lib/services/ciRunnerBootService.ts or lib/services/codeGraphIndexDispatchService.ts is modified.

Context refs

• lib/jobs/engine/step.ts — the shim, its JobStepYield contract, and the header that already argues WHY an in-process await was refused
• lib/jobs/engine/worker.ts — the lease, the heartbeat, the reclaim, and the attempt refund
• lib/jobs/engine/runner.ts — buildEngineContext, and that event.id is run.eventId
• lib/jobs/indexFleetSteps.ts — the loop being decided about, and dispatchId
• lib/services/codeGraphIndexDispatchService.ts — pollIndexContainer's deadlineVerdict, INITIAL_INDEX_POLL_STATE, INDEX_FLEET_TIME_BUDGETS
• lib/services/ciRunnerBootService.ts — FLEET_TIME_BUDGETS, runIntent, pollOnce
• lib/services/codeGraphIndexAdmissionService.ts — the slot and its already_held outcome
• docs/decisions/job-queue-foundation.md — the record this amends
• MOTIR-3422 — the shim card whose header this qualifies