MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3419

(motir-core) Decide the queue foundation — Graphile Worker vs pg-boss vs hand-rolled, judged against the step-shim requirement

Done
Description

Everything else in this story is shaped by this answer, so it is settled first, in writing, with evidence.

The question

What does the Postgres job engine build ON — Graphile Worker, pg-boss, or a hand-rolled queue?

The options, and the axis that actually separates them

Graphile Workerpg-bosshand-rolled
storagePostgres onlyPostgres onlyPostgres only
wake mechanismLISTEN/NOTIFYpolling + notifyours
cronbuilt inbuilt in (schedule)ours
debounce / singletonnoyes (singleton / throttle)ours
retries + backoffbuilt inbuilt inours
durable stepsnonono

The last row is the point. No candidate implements memoized steps or a durable sleep — so the step shim is OURS to build under every option, and the choice is only about what sits UNDER it: a claim loop, a schedule tick, and a retry policy. That reframes the decision from "which framework" to "how much of a well-understood loop do we want to own."

What to produce

A decision record under docs/decisions/ naming the choice and the evidence, structured as this corpus's other ADRs are. It must answer:

  1. The choice, with a rung-1 reading — what mature tools in this space actually do — rather than a preference.
  2. Whether the candidate's own tables can carry job_step, or whether we add our own beside them. A library whose schema we must extend is not obviously better than no library.
  3. How its claim loop interacts with FOR UPDATE SKIP LOCKED and with more than one worker, since that is what the worker subtask is built on.
  4. Its licence, checked from the installed package rather than the project page — the same discipline that found Inngest's server is SSPL while its SDK is Apache-2.0.
  5. What it costs to leave. We are migrating off a substrate right now because leaving was expensive; a candidate whose abstractions leak past lib/jobs/ repeats that.

Acceptance criteria

  • A docs/decisions/ record names the choice, the rejected options, and the reason each was rejected — with the reason for the winner stated as evidence, not preference.
  • The durable-step gap is stated explicitly: whichever is chosen, the shim is ours, and the record says so, so no later card assumes the library provides it.
  • The chosen package's licence is read from the installed artifact and recorded.
  • No candidate that requires Redis or any other new service is admitted — that constraint is the epic's premise and the record must show it was applied, not assumed.
  • The record names which subsequent subtasks it settles: the schema, the worker and the shim all read it.
  • No production behaviour changes in this card.

Context refs

  • lib/jobs/defineJob.ts — the option surface any candidate must be able to express (retryPolicy, concurrency, idempotency, debounce, cron)
  • lib/jobs/definitions/codeGraphRefresh.ts — the only debounce user, and the hardest option to reproduce
  • docs/decisions/job-lane-occupancy.md — why this migration is happening
  • docs/decisions/ — the ADR shape and header convention to follow