(motir-core) Prove the PILOT job on the engine in production — route `email.send`, read the run row back, and close MOTIR-3414's stated scope

Route the pilot job onto the Postgres engine in production and watch one run succeed. No PR — this is an operator action, marked done on Yue's confirmation.

The gap this closes, and why no existing card covers it

MOTIR-3414's scope boundary says, in its own words: "This story ENDS at: one pilot job running on the new engine in production, with the ledger, the DLQ and the operator dashboard behaving exactly as they do today."

That never happened, and the story is done. Verified from the running worker machine 8576143c4ee538 on release v131 (2026-08-25 19:27Z):

MOTIR_POSTGRES_JOB_IDS     = null

lib/jobs/engine/cutover.ts defaults every job to Inngest — deliberately: "the only way onto the new engine is for someone to name the job." Nobody named one. So the engine is built, merged, deployed and claiming an empty queue, and the story's stated end state is unmet.

No card owns it. MOTIR-3421 built the worker, MOTIR-3423 built the switch, MOTIR-3425 provisioned the machine — and none of them turns it on.

⚠️ MOTIR-3463 is NOT this card, and cannot substitute for it. That card cuts the 20 event-triggered jobs over in waves, and it is blocked_by MOTIR-3415 — a story that has not been built. Waiting for it would leave the engine unexercised in production for the whole of MOTIR-3415, which is exactly the risk MOTIR-3414 built a pilot to avoid. This card is the one job MOTIR-3414 promised, provable today, on a deployment that already exists.

Its wave 2 is email.send. When this card is done, that wave is a verification rather than a change — say so there rather than moving it twice.

Hypotheses to verify rather than assume

• The worker group is at count ≥ 1 and started. Read fly machine list -a motir-core --json, not fly.toml. As of 2026-08-25 it is: 8576143c4ee538, shared-cpu-1x/1 GB, started (MOTIR-3425). A queue with no worker accumulates rows silently and the dashboard shows nothing — indistinguishable from a job that never fired.
• MOTIR_POSTGRES_JOB_IDS is unset. Confirmed 2026-08-25 from inside the worker process. If it holds ids by the time this is picked up, stop and reconcile — someone has cut over ahead of this card.
• The queue tables exist in production. They ship in migration 20260823230609_add_postgres_job_engine_tables, applied by release_command on the v131 deploy. Confirm rather than assume: no job_queue_run table means the release did not carry it.

⚠️ The mechanism, and the way to get it wrong

cutover.ts reads the env var on every routing call rather than caching it, so no machine can hold a stale view. But an env var is still fixed at PROCESS boot: a new value takes effect when the machines restart, not when the secret is written. fly secrets set triggers a rolling update, which is what applies it. --stage defers it to the next deploy and would leave the change written and not live — do not use it here.

Rollback is the same door, and is the property the whole design was bought for: remove the id, set the secret again, one rolling restart. No deploy, no code change.

The steps

1. Re-verify the three hypotheses above from the platform.
2. fly secrets set MOTIR_POSTGRES_JOB_IDS=email.send -a motir-core — one id, the pilot, and nothing else.
3. Wait for the roll to finish (fly status), confirming the app group returns to 2 started.
4. Trigger the job for real: send a workspace invitation.
5. Read it back on /settings/workspace/jobs — a job_run row for email.send, succeeded, carrying its output, indistinguishable from an Inngest-backed run.
6. Confirm the email actually arrived. A green ledger row for an email nobody received is the failure this step exists to catch.
7. Exercise the rollback once: remove the id, set the secret again, confirm the next invitation runs on Inngest, then re-add it and leave it routed.

⚠️ What this actually changes in production

email.send is the invitation path — real mail to real recipients. This is the first production traffic the new engine has ever carried, so it is a risk decision, not a formality:

• it is the job carrying the only idempotency key, so the double-send failure mode is the one guarded best;
• fly secrets set rolling-restarts the app machines, which is a brief production restart in its own right;
• rollback is one command and takes effect on the next roll.

Pick a window where a failed invitation is recoverable, and prefer driving step 4 with an invitation you control rather than waiting for a customer's.

Acceptance criteria

• fly secrets list -a motir-core shows MOTIR_POSTGRES_JOB_IDS, and the machines were rolled after it was set — verified from the platform, never from a config file. The final value is recorded on this card verbatim.
• The value is exactly email.send — the pilot alone. The other 19 event ids and the 3 supervisor ids are NOT in it; those are MOTIR-3463's and MOTIR-3417's.
• A succeeded job_run row for email.send appears on /settings/workspace/jobs, quoted on this card — and the email it represents was received.
• The corresponding Inngest run shows { skipped: 'routed-to-postgres-engine' } rather than executing — the double-run guard, confirmed in production rather than in a test.
• The rollback was exercised once and the id restored, with what was observed recorded here.
• The app process group is unaffected and still at 2.
• This discharges MOTIR-3425's second criterion ("the pilot job's runs appear in the operator dashboard, proving the worker is claiming"), which that card cannot meet on its own. Note it there when this closes.

Worth taking while the pilot is live

With a job routed, a NOTIFY is finally sent — which makes claim latency measurable for the first time, and it is the direct falsification MOTIR-3454 is missing. Sub-second means the wake path works; ~5 s (IDLE_MAX_MS) means it is polling. Record whichever you see on MOTIR-3454. It does not gate this card.

Context refs

• lib/jobs/engine/cutover.ts — the env var, the live read, and the --stage hazard
• tests/e2e/jobs-postgres-engine.spec.ts:60 — const PILOT_JOB = 'email.send', which is where "the pilot job" is actually defined
• /settings/workspace/jobs — the operator surface this is read back from
• MOTIR-3429 — the planning bug about MOTIR-3414's post-deploy children; this card is a second instance of the same root cause