MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3471

(motir-core) The SCHEDULER — a tick in the worker that enqueues every due cron fire, with a guard that refuses to run against an empty registry

Done
Description

Build the half of the engine that does not exist: a tick that turns a cron expression into a job_queue row, and run it in the worker process.

What it does, per tick

For every definition in engineScheduledJobs() whose id routedToEngine(id) accepts:

  1. Compute the fires it owes, using previousFireAtOrBefore from lib/jobs/cron.tsthe evaluator is already written and tested (tests/jobs/cron.test.ts); do not write a second one.
  2. Apply that job's declared catchUp disposition to decide which of them to enqueue.
  3. jobQueueRepository.enqueueScheduled each one, with eventId: null, eventName: 'scheduled.<jobId>', runAt = the fire time, scheduledFor = the fire time, workspaceId: null, and maxAttempts from the definition. An already-queued result is the normal outcome, not an error — that is the whole point of the key, and it is what makes a second worker's tick a no-op instead of a duplicate run.
  4. notifyQueuedJob once if anything was enqueued, so a due run starts now rather than at the next poll boundary.

eventName is not cosmetic: jobRunsService writes it onto the ledger row through executeWithLedger, and jobScheduleHealthService groups on exactly scheduled.{functionId}. Get it wrong and every migrated cron job reads as overdue forever, which is the tripwire firing on the tripwire.

Where it runs, and the guard that makes that safe

In the worker process (scripts/worker.ts), beside the claim loop. That file already carries the side-effect import '@/lib/jobs/registry' with a comment explaining that the engine's tables hold only jobs whose definition module has been EVALUATED — and only two non-test files import that module (app/api/inngest/route.ts and scripts/worker.ts, git grep -l '@/lib/jobs/registry' -- '*.ts' '*.tsx' on origin/main@a26d500d).

So the scheduler must not merely rely on that import being present — it must fail loudly if it is not. engineScheduledJobs() returning an empty array in a process that believes it is scheduling is indistinguishable from "no cron jobs exist", and the resulting silence is exactly the failure MOTIR-3455 found on the emit path: nothing errors, nothing runs, and an operator sees a dashboard with no rows. Start-up refuses rather than proceeds.

Two properties to get right, both from the worker's own header

  • The poll is the correctness path. The tick is driven by the loop, not by a setTimeout chain that a paused event loop can skew. A tick that runs late still enqueues the fire it owed — the fire time is computed from the clock, never from "one interval since the last tick".
  • Shutdown is routine. SIGTERM arrives several times a day. A drain must not leave a half-enqueued tick, and it must not need the scheduler to be mid-sleep to exit promptly. Wire it into the existing drain rather than adding a second shutdown path.

Scope boundary

ENDS at: the scheduler enqueuing correctly for every routed cron job, its unit tests, and the worker wiring. Does NOT change claimDueRuns, the lease, the retry backoff, the ledger, the DLQ, or dispatchEventToEngine — a scheduled run is an ordinary job_queue row from the moment it is written, and everything downstream of the claim already handles it.

Does NOT touch defineJob's Inngest config or the serve route. A routed cron job still gets its Inngest tick and still returns { skipped: 'routed-to-postgres-engine' }; that guard already exists and is what stops a double run across engines. Verifying it holds for a CRON job specifically is the story gate's job.

Adds no environment variable. Routing is MOTIR_POSTGRES_JOB_IDS, which already exists; a scheduler that needed its own switch would be a second thing to get wrong during the production cutover.

Acceptance criteria

  • With a job's id routed, one tick at a moment its cron matches enqueues exactly one job_queue row carrying event_id = NULL, event_name = 'scheduled.<jobId>', scheduled_for = the fire instant, and the job's maxAttempts.
  • With the id NOT routed, the tick enqueues nothing for it — the switch's default is Inngest and the scheduler must respect it exactly as dispatchEventToEngine does.
  • Ticking twice for the same fire produces one row, and the second tick reports already-queued rather than throwing.
  • Each catchUp disposition is honoured, driven by an injected clock: for a job whose last enqueued scheduled_for is N fires behind, the row set the tick produces matches what that job declared — every missed fire, only the most recent, or none.
  • The scheduler refuses to start when engineScheduledJobs() is empty, with an error naming the missing registry import. Asserted by a test that constructs it without the import.
  • A tick that arrives late does not shift the schedule: fire times come from the cron expression evaluated against the clock, never from the previous tick's timestamp.
  • SIGTERM drains cleanly with the scheduler running — no half-enqueued tick, and the existing drain assertions in tests/jobs/engine-worker.test.ts still hold.
  • now is injectable throughout, as jobScheduleHealthService.check(now) already is, so tests pin a moment rather than racing the wall clock.

Context refs

  • lib/jobs/cron.tsparseCron, previousFireAtOrBefore, CRON_SEARCH_HORIZON_DAYS, and the UTC rule in its header
  • lib/jobs/engine/registry.tsengineScheduledJobs(), its first real caller
  • lib/jobs/engine/dispatcher.ts — the enqueue shape and the already-enqueued-is-not-an-error precedent to mirror
  • lib/jobs/engine/worker.ts — the loop, the drain, notify(), and the poll-is-correctness / NOTIFY-is-latency argument
  • lib/jobs/engine/notify.tsnotifyQueuedJob
  • scripts/worker.ts — the process, and the side-effect import comment
  • lib/jobs/engine/cutover.tsroutedToEngine, read live per call
  • lib/services/jobScheduleHealthService.ts — why event_name must be exactly scheduled.{functionId}
  • tests/jobs/engine-worker.test.ts · tests/jobs/cron.test.ts — the suites to extend