MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3483

(motir-core) The engine's DEBOUNCE — a same-key `run_at` pushed forward on each arrival, so a push burst still coalesces into ONE run carrying the latest event

Done
Description

Implement defineJob's debounce option on the Postgres engine, so a burst of default-branch pushes to one repo still produces ONE system.code-graph-refresh run carrying the LATEST push.

This is the whole of the story's cutover half. The engine's step shim already implements step.run and step.sleep faithfully (lib/jobs/engine/step.ts, whose header names indexFleetSteps and ciRunnerFleet as the reason it was built that way), so the three supervisors would otherwise run on the engine with no code change at all once the emit seam lands. debounce is the one defineJob option they use that the engine does not have.

What is missing, verified on origin/main@7e97e2ed

  • lib/jobs/defineJob.ts accepts debounce?: { key: string; period: string; timeout?: string } and forwards it into the Inngest config object. registerEngineJob is called with { id, trigger, cron, maxAttempts, retryPolicy, handler } and nothing else, and EngineJobDefinition in lib/jobs/engine/registry.ts has no such field.
  • dispatchEventToEngine (lib/jobs/engine/dispatcher.ts) writes a job_queue row with runAt: new Date() for every subscriber, unconditionally. Nothing reads a debounce.
  • system.code-graph-refresh is the ONLY job in the tree declaring one: key: "event.data.installationId + '/' + event.data.repoOwner + '/' + event.data.repoName", period: '2m', timeout: '15m' (lib/jobs/definitions/codeGraphRefresh.ts).

The shape to build — the mechanism is already DECIDED, not open

docs/decisions/job-queue-foundation.md §9 chose it when it rejected pg-boss, and this card implements that sentence rather than re-opening it:

"Its semantics are 'hold until period passes with no further same-key event, then run once with the latest' — a run_at that is pushed forward on each same-key arrival, which is a column and an upsert on a table we own, not a subsystem."

  1. Carry the option through registration. EngineJobDefinition gains debounce, passed by defineJob from the same object Inngest already receives — one field, at the choke point every job passes through, exactly as MOTIR-3459 does for idempotency.
  2. Resolve the key. ⚠️ The debounce key is a CONCATENATION, and MOTIR-3459's resolver handles the single event.data.<field> form and THROWS on anything else. So this card WIDENS that resolver rather than adding a second one — support a +-joined sequence of event.data.<field> terms and single-quoted string literals, keep the throw-on-anything-else totality, and keep one resolver for both options. Reproduce the ADR's own note that an unresolvable key MERGES rather than disabling the debounce only if you choose to reproduce it; the safer engine behaviour is to refuse at registration, since our resolver runs at declaration time and Inngest's ran per event.
  3. Denormalise and upsert. job_queue gains a debounce_key, and the enqueue becomes: if a pending, unclaimed run exists for (job_id, debounce_key), push its run_at to now + period and REPOINT its event_id to the new event (this is what makes the coalesced run carry the latest push); otherwise insert one. ⚠️ That is a read-derived writeSELECT … FOR UPDATE the candidate row inside the same transaction before updating it, per motir-core/CLAUDE.md's lock-before-a-contended-update contract; a plain read-then-write lets two concurrent pushes both insert. A partial unique index on (job_id, debounce_key) WHERE debounce_key IS NOT NULL AND state = 'pending' is the constraint that makes the race outcome recoverable rather than silent, and Prisma cannot express a partial unique index, so it is raw SQL in the migration with a prisma migrate diff check afterwards.
  4. Honour timeout, or say why not. timeout: '15m' is meant to cap total deferral. MOTIR-2994 MEASURED that Inngest's cap does not fire for a stream faster than ~1 event/second, and §9 says explicitly "a property of Inngest's implementation that we are free not to reproduce." Implementing it correctly here is cheap — stamp the first arrival and refuse to push run_at beyond first + timeout — so implement it, and record in the PR body that the engine's cap is honoured where Inngest's was not. A deliberate divergence stated on the day it is made, not discovered later.

Scope boundary

ENDS at: a job declaring debounce and routed to the engine coalescing a same-key burst into one pending run carrying the latest event, with tests against real Postgres.

Does NOT move any job onto the engine. MOTIR_POSTGRES_JOB_IDS is untouched — the production flip is the epic-level operator task.

Does NOT change codeGraphRefresh's declared key / period / timeout, its handler, or its Inngest behaviour — the config object defineJob builds for it must be unchanged, asserted off fn.opts.

Does NOT implement concurrency on the engine. No job in this story's set declares one, and codeGraphRefresh's own header argues at length that a cap on a container supervisor caps supervisors rather than containers. If a later job needs it, that is its own card.

Does NOT touch the supervision loopsthe index collapse and the CI collapse own those files.

Acceptance criteria

  • A job declaring debounce and routed to the engine, sent N same-key events in a burst, holds exactly ONE pending job_queue row, whose event_id is the LAST event sent and whose run_at is period after that last arrival — asserted against real Postgres, not a mock.
  • Two DIFFERENT keys produce two rows, and a job declaring no debounce produces one row per event exactly as today.
  • At least one test drives the burst CONCURRENTLY on a warm pool — two dispatches racing on the same key — and the outcome is one row, never two and never a thrown error reaching the caller.
  • A run that has already been CLAIMED is not coalesced into: a same-key event arriving while the debounced run is executing enqueues a NEW run, so a push during an index is not silently dropped.
  • The key resolver returns the same string for codeGraphRefresh's declared expression as Inngest's would for the same payload, asserted on a real CodeGraphRefreshData; and it THROWS at registration on an expression it cannot resolve, asserted by a test registering one.
  • run_at is never pushed past first_seen + timeout, asserted by a test whose burst outlives the window.
  • prisma migrate diff reports no drift between prisma/schema.prisma and the migrated database after the raw-SQL partial index lands.
  • codeGraphRefresh's Inngest configuration is byte-identical, asserted off fn.opts — the existing assertion in tests/jobs/code-graph-index.test.ts still passes untouched.

Context refs

  • lib/jobs/defineJob.ts — the debounce option, and where it is forwarded to Inngest and dropped for the engine
  • lib/jobs/engine/registry.tsEngineJobDefinition, the field to add
  • lib/jobs/engine/dispatcher.ts — the enqueue, and the P2002-as-success pattern to reuse
  • lib/jobs/definitions/codeGraphRefresh.ts — the only declaration, and the warning that a key naming an optional field would merge unrelated repos
  • prisma/schema.prismaJobQueueRun, its run_at comment ("a step.sleep re-enqueue moves it forward") and its @@unique([eventId, jobId])
  • docs/decisions/job-queue-foundation.md §9 — the decision this implements, quoted above
  • docs/jobs.md § Debounce — MOTIR-2994's measurement table, and the timeout limit this card is entitled to improve on
  • tests/jobs/debounce-burst.test.ts — the Inngest-side guard, which boots the real dev server; this card's engine twin is its sibling, not its replacement
  • MOTIR-3459 — the resolver and the denormalised-key pattern this widens rather than duplicates