[cb1kenobi]

⊙ Problem

A job runs in its own worker thread, and that worker writes the job's final status. If the thread dies first while Harper keeps running, get_job reports the job IN_PROGRESS until the next restart. The boot sweep from Settle jobs left unfinished by a Harper process that is gone (#2645) skips rows the running process owns. @kriszyp raised this gap there.

❓ Your call: is settling in-process worth it?

I think yes: without it, a dead job worker reads as running until a restart. The alternative is to accept that and rely on the boot sweep alone, which means reverting this PR.

💡 Solution

• When a job worker exits and its row is still CREATED or IN_PROGRESS, the row becomes ERROR with "Job was interrupted: its worker thread exited before it finished".
• A rolling restart does not settle the row, because its replacement worker re-runs the job. Process shutdown leaves the row to the boot sweep.
• Not fixed here, and older than this PR: a dev-mode reload re-runs a running job, and can overwrite COMPLETE with ERROR. The design note records it.

⚖️ Alternatives

• worker.on('exit') in the job runner — a replacement started by startCopy() would not inherit it. The hook is a startWorker option instead.
• Settle on every exit, including deliberate stops — briefly marks a job ERROR that a replacement is about to re-run.

❓ Your call: keep onPermanentStop?

Nothing stops a job worker for good while Harper keeps running today, so the option has no production trigger. I kept it because restartWorkers without replacements can reach it. Dropping it is a two-line change.

🔧 Changes

• server/threads/manageThreads.ts — startWorker takes onUnexpectedExit and onPermanentStop options. exitedUnexpectedly() routes each exit, and the restart decision re-reads it after the hook. runExitHandler catches a handler that throws or rejects. A permanent stop is marked in stopWorker and in restartWorkers without replacements. whenWorkerStarted captures the thread id before it reads -1.
• server/jobs/jobRunner.ts — both launch paths (main thread, START_JOB) use startJobWorker(). Its options come from the exported jobWorkerOptions(), so tests can call the hooks without rewire or sinon.
• server/jobs/jobOwnership.ts — settleAbandonedJob() re-reads the row and settles it only if it is unfinished. The boot sweep shares settle() and jobTable(). Its ownership filter is unchanged.

❓ Your call: retry a failed settle?

A failed settle is logged, and the boot sweep claims the row later. One delayed retry would cover a transient 503. I left it out to keep the exit path simple.

• server/jobs/DESIGN.md (new), indexed in DESIGN.md — which mechanism settles a row depends on what survived: the process or only the thread.
• DESIGN.md, server/DESIGN.md, resources/DESIGN.md — reflowed, no wording changes, to meet the line budgets main currently fails. The resources/ change is identical to Bring resources/DESIGN.md back under its line budget by reflowing one hard-wrapped note #3802. Easy to drop if you'd rather land it separately.

✅ Verification

• unexpectedExit.test.js kills a real worker thread (fixture). The hook fires on a crash, on a startCopy() replacement, and not on a deliberate stop. A throwing or rejecting handler is contained.
• jobOwnership.test.js settles real job rows, and runs both job-worker hooks. Deleting the hooks from dist fails those two tests.
• After npm run build: 45 passing in these files and jobRunner; 72 in the other server/jobs and manageThreads tests; 137 passing, 7 pending in the restart-path thread suites. Lint, format, check:design-docs and tsc on touched files are clean.
• Not run: full test:unit:main since the merge with main, and an end-to-end test of a real job worker dying, which no operation can trigger.

🤖 Generated with Claude Code

Release target: main — milestoned v5.4 (the main line); a v5.3 backport of #2645 follow-ups is a customer-need call, not requested
Related PRs: #3802 overlaps (identical resources/DESIGN.md reflow; either merge order applies cleanly), #2752 overlaps (DESIGN.md / resources/DESIGN.md index and note lines only), #3763 independent, 19 others independent
Complexity: medium

_{Review-Coverage: authored=claude; ran=gemini,codex; adjudicated=domain; blocked=cursor-composer(auth); declined=cursor-grok,cursor-kimi,cursor-muse; rounds=3; full=1 @ 9147a6f}

_{Review-Attention: study ~9m (hot: manageThreads.ts; decisions: do-less-alternative, settle-failure-deferred-to-boot, test-seam-export, rolling-restart-reruns-job, design-budget-reflow-in-pr) @ 9147a6f}

[claude]

Reviewed; no blockers found.

[kriszyp]

Great, good approach!

_{🤖 Reviewed with Codex}

[kriszyp]

These APIs accept a worker type for a permanent scoped stop: for example, shutdownWorkersNow('job') marks the job worker as shut down, does not start a replacement, and leaves processShuttingDown false. This condition then skips settlement, while the boot sweep cannot run until a later process start, so the row remains CREATED or IN_PROGRESS in the live process. Please settle the associated job on permanent targeted stops, or prevent these helpers from stopping job workers without a replacement or process teardown.