MotirBuilding in public
MOTIR · moooon
onMotir
You’re viewing a public project. Anyone can view it — no account needed. Sign in to submit, upvote, or comment on requests.View-only — you can’t edit work items
MOTIR-3424

(motir-core) Retries, `onFailure` and DLQ parity — the `job_run` ledger and the operator dashboard keep their shape

Done
Description

Makes failure on the new engine behave exactly as it does today, so that the operator surface shipped in MOTIR-64 keeps working without being touched.

What must be preserved, not reinvented

  • Named retry policies. retryPolicy: 'idempotent' and 'transient' keep their current attempt counts and backoff. A job that retries five times today retries five times after this card.
  • The job_run ledger. One row per run: running at start, succeeded on return with the handler's JSON-safe output, failed once the budget is exhausted. The row must be written exactly once per run even when the handler replays across step boundaries — the existing implementation gets that by writing it inside a memoized step, and the same trick applies here.
  • The DLQ. On terminal failure a dead-letter row is written and the run is replayable from /settings/workspace/jobs.
  • onFailure semantics. The current code moved the dead-letter write OUT of a try/catch into Inngest's onFailure hook for a specific reason: on the real executor, a step.run scheduled from a catch block after the terminally-failing step never executes, so the failed/DLQ rows silently never got written in production while the in-process test harness made it look like they did. The Postgres engine must provide an equivalent after-all-retries-exhausted hook, and the reason must be recorded — otherwise the next person simplifies it back into a catch and reintroduces a bug that only appears in production.

Acceptance criteria

  • A job exhausting its retries writes a failed job_run row and a DLQ row, asserted against a real Postgres by actually exhausting the budget rather than by calling the hook directly.
  • Replay from the operator dashboard re-runs the job and succeeds — the existing UI works unchanged against the new engine.
  • Exactly one job_run row per run, including when the handler replays across steps.
  • A non-JSON-safe return degrades to a NULL output rather than failing the run, as it does today.
  • The onFailure-not-catch reasoning is carried in a comment on the new implementation, citing why the catch-block form fails only in production.
  • Unit and real-Postgres integration tests ship with the change.

Context refs

  • lib/jobs/defineJob.ts — the ledger writes and the onFailure wiring
  • lib/jobs/dlq.ts · lib/jobs/retries.ts — the DLQ and the named policies
  • lib/services/jobRunsService.ts · app/(authed)/settings/workspace/jobs/page.tsx — the operator surface that must not change
  • docs/jobs.md § Run ledger — the contract to preserve