MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3682

(motir-core) A 15th scheduled job landed mid-cutover and is still on Inngest — `system.public-follow-digest-tick` breaks MOTIR-3418's "nothing remains on Inngest" premise

Done
Description

Found while running MOTIR-3475. Nothing is broken — this is the cutover switch's safety default behaving exactly as designed. The defect is a COUNTING one, and it lands on a card downstream.

What happened

MOTIR-1103 (#2344, the public changelog + weekly digest) merged on 2026-08-26 evening and added a new scheduled job:

lib/jobs/definitions/publicFollowDigestTick.ts:32
  id:   'system.public-follow-digest-tick'
  cron: '0 9 * * 1'            (weekly, Mondays 09:00Z)
  retryPolicy: 'idempotent'

git grep "cron: " lib/jobs/definitions | wc -l now returns 15.

MOTIR-3475 — authored 2026-08-25, executed 2026-08-26/27 — says "the 14 scheduled jobs" in its title, its body and every wave list, and its final MOTIR_POSTGRES_JOB_IDS routes 14. The 15th arrived between the card being written and being finished, so it was never in scope and is not in the routed set.

It therefore runs on Inngest. That is lib/jobs/engine/cutover.ts working as documented — "a job absent from the configuration runs where it runs today … a job nobody has thought about cannot be silently migrated". The default is the safety property, and it held.

⚠️ Why it still needs a card

MOTIR-3418"Retire Inngest — delete the SDK, the serve route and the dependency, and prove the substrate stands alone"is premised on nothing remaining on Inngest. Today exactly one thing does, and no card owns moving it. ⚠️ FALSE — see the AMENDMENT below: three did. Deleting the SDK and serve route with system.public-follow-digest-tick unrouted would stop it firing entirely, silently, with no error: its cron would have no timer on either lane.

The window is small and the consequence is quiet, which is the combination that survives review.

Fix direction

Add the id to MOTIR_POSTGRES_JOB_IDS and read one tick back from the ledger. It is idempotent and weekly (0 9 * * 1), so the next natural fire is Monday 2026-08-31 09:00Z — but routing it will also trigger one immediate catch-up run of the last missed slot (the property recorded on MOTIR-3475), which is a perfectly good first observation and is safe here.

Also worth deciding, and the more valuable half: nothing detected this. A job was added to the codebase while a migration of exactly that population was in flight, and it was found by a human reading a log line. A guard asserting every job declaring a cron appears in the routed set, or is on a named exclusion list would have failed the moment #2344 merged. tests/jobs/scheduled-cutover-story-gate.test.ts already exists and is the natural home.

Acceptance criteria

  • system.public-follow-digest-tick is in the live MOTIR_POSTGRES_JOB_IDS, verified from inside a machine, and the full value is recorded verbatim.
  • One engine-lane succeeded run is quoted from job_run, with its tick. ⚠️ NOT MET at the time of writing — blocked on a DEPLOY, not on routing. See the amendment.
  • The three container-supervisor ids are still absent (this card must not widen the set beyond its one id).
  • A test fails when a job declaring a cron is neither routed nor on an explicit exclusion list — or a recorded decision says why that guard is not wanted.
  • MOTIR-3418 can state "nothing remains on Inngest" and have it be true. ⚠️ AMENDED — not reachable from this card alone; see below.

Context refs

  • lib/jobs/definitions/publicFollowDigestTick.ts — the new job
  • lib/jobs/engine/cutover.ts — the default-to-Inngest safety property that made this quiet rather than broken
  • tests/jobs/scheduled-cutover-story-gate.test.ts — the natural home for the guard
  • MOTIR-3475 — routed 14 of 15; its record explains why

⚠️ AMENDMENT — 2026-08-27, during the run that executed this card

The enumeration above was wrong when it was written, and it is short in the direction that matters. Recorded here rather than silently re-scoped (run.md's a falsified COUNT is REPORTED, never silently re-scoped). Planning bug: MOTIR-3689.

The measurement. On origin/main at 555932db9, against the live MOTIR_POSTGRES_JOB_IDS read from inside motir-core machine 8576143c4ee538 at 2026-08-27T10:31Z:

git grep -hoE "id: '[^']+'" -- lib/jobs/definitions | sed "s/id: '//;s/'//" | sort -u   →  40
live MOTIR_POSTGRES_JOB_IDS                                                            →  34
                                                                                unrouted:  6
unrouted idkindowner
system.ci-runner-bootcontainer supervisorMOTIR-3489
system.code-graph-indexcontainer supervisorMOTIR-3489
system.code-graph-refreshcontainer supervisorMOTIR-3489
system.public-follow-digest-tickscheduledthis card
plan-drift/transitionedevent-triggerednobody → filed as MOTIR-3688
public-follow/digestevent-triggerednobody → filed as MOTIR-3688

Both extras PREDATE this card, so this is an enumeration that was wrong, not drift since: plan-drift/transitioned merged 2026-08-26 20:39Z (#2309), public-follow/digest merged 2026-08-27 07:07Z (#2344 — the same pull request that added the job this card is about), and this card was authored 2026-08-27 08:54Z.

Root cause: the count was taken with git grep "cron: " — one LANE — and the sentence built on it quantifies over the whole population. The command is correct; it answers a narrower question than the claim.

What changes on this card:

  1. AC 5 is not reachable from here and is not this card's to reach. It now reads: the census's Inngest side is empty, which needs this card and MOTIR-3489 and MOTIR-3688. All three now blocks MOTIR-3418, so the premise is enforced by the graph rather than by a sentence.
  2. AC 4's guard is widened from cron-declaring jobs to the WHOLE registry, and lives in its own file rather than the MOTIR-3416 story gate this card nominated. Two of the three misses were event-triggered: a guard cut to the population the defect was first noticed in would have been green through both of them. tests/jobs/cutover-census.test.ts ⚠️ CORRECTED — the file is tests/jobs/every-job-declares-its-lane.test.ts (#2348), with MIGRATED_TO_ENGINE / DELIBERATELY_ON_INNGEST. Two sessions ran this card concurrently and each built the same guard; the one that shipped is #2348's, and the duplicate was discarded. Read AC 4 against that file. See the run-report comment below.
  3. AC 1–3 are unchanged and are delivered as written — one id added, the three supervisor ids untouched.

⚠️ AC 2 IS OUTSTANDING — do not read the merge of #2348 as satisfying it

Routing is live and correct; the job is not in the deployed image, so it has no timer on either lane and job_run / job_queue are both empty for it. FLY_IMAGE_REF on all four machines is release v166, 2026-08-27T01:50Z; #2344 merged at 07:07Z; v167–v172 are secrets set restarts on that same image. main has not deployed because its CI is red — filed as MOTIR-3692.

Before closing this card, re-read job_run for system.public-follow-digest-tick after the next real deploy. catchUp: 'latest' enqueues the missed 2026-08-24T09:00Z slot on the first scheduler tick after it, so the observation arrives on its own — it just has not arrived.