[kriszyp]

⊙ Problem

With threads.maxIsolated: 1, an isolated application whose first release fails its canary blocks every other isolated deploy with a 409 until it is redeployed or dropped. Deploy admission counted every isolated application on disk; the reconcile that starts workers skips a failed-closed release. They also disagreed when boot order would not admit an application on disk: it deployed, then nothing started it.

🤔 Is the requirement right?

Yes. Admission now follows boot order, so some deploys that used to go live with no worker get a 409. Which application a scoped reconcile should prefer stays open.

💡 Solution

• Admission and the reconcile ask one selector, admittedIsolatedApplications. A failed-closed release holds no place.
• Admission answers 409 whenever the reconcile would refuse the deploying application.
• It still counts workers a scoped reconcile leaves running, unless the rejection of an application still isolated withdrew the worker (slotWithdrawn); look hardest here.
• Reactivating a refused live release requires admission first.

⚖️ Alternatives

• Filter failed-closed releases from the old count — keeps two computations that still disagree about present applications.
• Let main answer admission — main's cached config can lag, and refreshing it there can rewrite the config file.
• Delete a failed first release — certification keeps it live and failed closed on purpose.

🤔 A deploy on a worker thread reads refusals from disk only

The operations API runs on main, which sees every refusal. A worker's server.operation deploy misses refusals main holds only in memory. This can only refuse more, never admit what the reconcile refuses; closing this gap needs a second field on the topology reply.

🤔 An isolated deploy reads refusals twice

Kept: the pre-check refuses before staging; the locked check is authoritative. Cost: one directory read per deploy.

🔧 Changes

• server/threads/isolatedApplications.ts — the selector and failed-closed reader, moved from socketRouter.ts, plus isolatedApplicationAdmissionRefusal for deploys.
• server/threads/socketRouter.ts — boot and reconcile call the shared selector and keep their refusal reporting.
• components/operations.ts — assertIsolationAdmission uses the shared admission and answers 503 when refusals cannot be read; admitIsolation without a descriptor admits the live entry and checks a component grant against it.
• components/Application.ts — recertifyLiveRelease admits before it arms a refused release; admitIsolation's descriptor is optional.
• components/releaseCertification.ts — RejectedReleaseError.slotWithdrawn, set only for a rejection with nothing to restore, and for a refusal main holds in memory.
• components/DESIGN.md — the shared selector and the occupancy count.
• Refs Start and stop one isolated application without restarting the shared workers #3339, which added the scoped reconcile.

✅ Verification

• New maxIsolated: 1 integration suite: after A's first release fails closed, B deploys and serves through its mirror. A's fix gets 409 while B holds the place, and is certified once B is dropped. On base, B's deploy gets the 409. Without the new admission call, the same-ID activation answers 200 uncertified.
• Gates: test:unit:main 8323 passing; test:unit:resources 4898 passing; test:integration:all 2776 passing (four ollama-backend failures: no local models).
• PR QA: worker-thread deploy and restart after a failed first release — queued @ e50e7f8, qa-pr-harper-3607-e50e7f8; later commits touch no path it reaches.
• Docs: queued docs-harper-3607 for reference/components/applications.md.
• Late Grok findings below predate the occupancy, 503 and comment fixes.

Unit test cases

• unitTests/server/threads/isolatedApplications.test.js: running ones keep their place; config order up to the budget; unreadable refusals admit nothing new; a failed first release frees its place; a candidate's own refused release is ignored but a full budget still refuses it; two present with a budget of one; placement of a new integer-like name, a new name and a flip to isolated; occupancy counts every running worker unless its refusal withdrew the slot, and counts an application turned shared; a budget of zero.
• unitTests/components/releaseCertification.test.js: only a rejected release with nothing to restore reports slotWithdrawn.
• unitTests/components/deployCertification.test.js: re-arming a rejected live release calls admitIsolation with no descriptor, arms nothing when refused, and leaves the record rejected.
• integrationTests/components/isolated-application-lifecycle.test.ts: the existing suite now shares its helpers with the new one.

_{🤖 Generated by Claude Code (Claude Opus 5.5); posted via @kriszyp.}

Release target: main — release certification, which leaves a failed first release in place, exists only on main
Related PRs: #3505 overlaps (this branch is rebased on it; both edit the admitIsolation callback), 1 other independent
Complexity: complicated

Deferred findings

• 3af96fffdf79ff4d minor concurrency — components/operations.ts — Isolated admission is per-component locked, so concurrent deploys of different apps can both pass — NOT FILED
• fb2f1c1838d28ea0 minor correctness — components/operations.ts — Worker-originated admission omits main's memory-only release refusals — NOT FILED
• 7a35bd5966585ca9 nit correctness — server/threads/isolatedApplications.ts — Disk refusal with slotWithdrawn false masks main's withdrawn in-memory entry — NOT FILED
• 5ace95afe0f4a66d nit reliability — server/threads/isolatedApplications.ts — Running refused or no-longer-isolated workers hold no selector place; budget checked in two parts — NOT FILED
• d1b1df8a04d8df60 nit performance — components/operations.ts — Isolated deploy admission awaits a staging-root refusal scan, more than once per deploy — NOT FILED
• 297fa6f6c1579f90 nit performance — server/threads/isolatedApplications.ts — admitted.includes in loop is quadratic — NOT FILED
• 1168f9f920830637 nit maintainability — components/operations.ts — Isolation admission errors use banned handleHDBError instead of ClientError/ServerError — NOT FILED

Late Grok findings

4 surviving (highest major) — Grok reviewed 5651bea

• major — server/threads/isolatedApplications.ts:136 — capacity counting drops every running worker whose release is failed-closed, including one that is failed-closed only because its certification record became unreadable (components/releaseCertification.ts:225-231). Nothing queues that worker's withdrawal, so it keeps serving while B is admitted un
• minor — components/operations.ts:588 — the new failedClosedApplications() failure branch uses legacy handleHDBError. AGENTS.md forbids that helper, and HdbError exposes the raw filesystem message instead of the deployment-context text; should be new ServerError(msg, 503)
• nit — components/operations.ts:571 — an isolated deploy scans .deploy-staging for refusals twice: once in the pre-check at :607, and again when admitIsolation re-runs admission under the preparation lock (components/Application.ts:5287). The cost lands on the deploy path, not a request path
• nit — integrationTests/components/isolated-application-lifecycle.test.ts:85 — added comments that narrate adjacent code, here and at components/Application.ts:5286 and unitTests/server/threads/isolatedApplications.test.js:125-128. Also, the comment at unitTests/components/deployCertification.test.js:307 says "join", but the code it describes is the optional-join fallbac

Not standing: 3 separate-issue. Grok ran 24m (24m from launch to report); the fold adjudication (cursor-grok@5651beaf1202-6452690e, claude) took 1m.

Origin — the dispatch brief this PR was written from

Isolated-application admission is computed in two places that disagree: assertIsolationAdmission (components/operations.ts) counts a failed-closed first release (directory and config entry left on disk) against maxIsolated, while admittedIsolatedApplications in server/threads/socketRouter.ts excludes it. With maxIsolated: 1, a failed first deploy of app A makes a deploy of app B return 409 until A is redeployed or dropped. Export one admitted-set function (socketRouter.ts or isolatedApplications.ts), failed-closed excluded, and have assertIsolationAdmission use it. Do not change scoped reconcileIsolatedWorkers ordering semantics (boot order vs requested app) in this PR; that is an open design question.

Acceptance

Test: maxIsolated: 1, deploy app A with a failing first release, then deploy app B -> B succeeds; existing isolation tests green. Show each new test failing on origin/main sources (forced rebuild) before the fix.

_{Dispatch: task harper-isolation-admission-shared-set · queued by finding-triage/wave-40 · ran by claude/opus/xhigh · worker kzyp-xps-1}

_{Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; declined=muse; rounds=8; full=2 @ ff2525b}

_{Review-Attention: study ~15m (hot: isolatedApplications.ts, socketRouter.ts; decisions: slot-withdrawal-source, admission-two-checks, recertify-admits, do-less-alternative) @ ff2525b}

[gemini-code-assist]

Code Review

This pull request refactors and improves the admission logic for isolated applications under the threads.maxIsolated budget, ensuring that only applications with a dedicated worker are counted. It introduces helper functions like admittedIsolatedApplications, isolatedApplicationAdmissionRefusal, and failedClosedApplications to centralize this logic, and updates deployment and reconciliation processes. Feedback on the changes identifies an issue where in-memory refusals are instantiated as plain Error objects instead of RejectedReleaseError, which prevents them from being recognized as having their slots withdrawn and incorrectly charges them against the budget.