Skip to content

feat(directsdk): one kept Claude Code process per agent: fails closed, tested versions only, started only where it pays - #402

Open
metahacker wants to merge 155 commits into
NousResearch:sid/ddsdk-transportfrom
metahacker:feat/directsdk-session-steer
Open

metahacker wants to merge 155 commits into
NousResearch:sid/ddsdk-transportfrom
metahacker:feat/directsdk-session-steer

Conversation

@metahacker

@metahacker metahacker commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #361. One kept Claude Code process per Hermes agent, with what the maintainer review asked for before it lands: main merged in, fail closed, a version gate, settings in user config, and memory that follows the benefit. Carries the fork branch the review asked for (feat/directsdk-session-transport, its 20 commits unchanged) and steering on the kept process.

The gain is latency and the host's CPU time. Nothing changes upstream: a kept process's request goes upstream only as the per-call path's bytes: where the relay cannot make it so, it refuses before upstream and the step replays on a new process, so cost and caching do not change. An earlier version of this text said every request was byte-identical. At fc71f5d that was not true for some histories. They are listed with their fixes here: #402 (comment)

The long session

evals/directsdk_session_bench.py, real Claude Code against a loopback upstream, a first turn of 200 tool rounds with a /steer after rounds 50 and 200, then a compaction and an edited tool result, each with a new turn, at 8411b10. Median seconds per model step:

Linux 2.1.292 Linux 2.1.326 macOS 2.1.292 macOS 2.1.326
Per-call, rounds 0 to 49 0.445 0.423 0.191 0.177
Per-call, rounds 150 to 199 0.858 1.025 0.332 0.357
Kept, rounds 0 to 49 0.024 0.026 0.010 0.010
Kept, rounds 150 to 199 0.039 0.041 0.018 0.018
/steer after round 200: kept / per-call 0.049 / 0.950 0.053 / 1.181 0.021 / 0.352 0.025 / 0.385
Whole scenario: per-call / kept (s) 138.7 / 8.7 148.4 / 9.4 54.7 / 4.2 55.9 / 4.4

Linux: AWS, 16 vCPU, x86-64. macOS: Mac mini, Apple M4, macOS 26.6.2. Linux ran every listed version (2.1.322 to 2.1.325 too, 2.1.327 at 8631b5f and 2.1.328 at 0f35b81); every request of every kept run was byte-identical to per-call's. At 74a5ae5 (main 72eb3b8 merged) the bench ran again on 2.1.292 and 2.1.328: every kept request byte-identical to per-call's (209 Claude Code processes per-call, 6 kept); its timings are not used, the host was not quiet. The bench's replies carry the caller the Messages API puts on each tool call, as live replies do.

CPU, at 0f35b81, Claude Code 2.1.327, the same bench at 100 rounds with no steer and session_min_rounds 0, getrusage(RUSAGE_CHILDREN) and time.process_time() read around each mode. One run, the Linux host under its normal load: per-call 81.6 s (74.7 s in 109 Claude Code processes, 6.9 s on the Hermes side), kept 7.1 s (5.3 s in 6 processes, 1.8 s); every kept request byte-identical to per-call's. RomainNicault measured this first, on a 4 vCPU host: 188.8 s against 13.8 s (#402 (comment)).

Live, a claude.ai Pro login, Opus 5.5, Hermes v0.23.1, Claude Code 2.1.322, the plugin's defaults: 34 tool rounds, a /steer at the 12th; kept at 8411b10, per-call at 74accf4 (the per-call path is the same code). Model calls from step 11 on: 1.00 s median kept, 1.50 s per-call; the steer call 1.8 s against 1.5 s; the whole run 50.4 s against 58.5 s. In both modes each step reads the previous request from the cache less 2 to 4 tokens (per-call 317 at step 2), and the steer step writes 228 tokens. No compaction ran this time (see #402 (comment)). One run per mode: an example, not a measurement. The run of fc71f5d, with a compaction: #402 (comment)

What is in

  • Kept process (feat(directsdk): one Claude Code process per agent #361): Hermes' tools are an SDK MCP server inside the plugin; a tool call waits on Claude Code's control channel until Hermes brings the result.
  • Review fixes for the earlier reviews: the cap is taken under the lock; a stop during the hand-over ends the process; bounded tool-result writes; a dead idle process gives its place back; a dropped client ends its process; launch-directory cleanup; the session tests run in evals/ci.sh.
  • Steering: a /steer after a tool round continues the same process. Claude Code's own steer row is put back where Hermes has it, so the bytes are per-call's.
  • Main merged in (feat(directsdk): cap concurrent Claude Code processes #367, then claude-subscription-directsdk: land 15 reviewed contributor fixes (#426 #382 #406 #421 #422 #423 #427 #425 #419 #416 #412 #404 #408 #410 #387) #445 at 72eb3b8): a kept process holds a max_concurrent slot only while it serves a step. Main's claude-subscription-directsdk: land 15 reviewed contributor fixes (#426 #382 #406 #421 #422 #423 #427 #425 #419 #416 #412 #404 #408 #410 #387) #445 fixes reach the kept process too; where it missed one, it is fixed here (the merge comment has the table).
  • Fail closed: rows that do not line up, or a body the relay cannot read, are refused before upstream (HERMES_SESSION_DIVERGED). A new process's first request is the replay as per-call builds it. A kept process's first resumed request goes upstream only if it equals the request a per-call step builds for the same history. At each later step, Claude Code's record of each new reply and each tool result must be Hermes' frame byte for byte (less the "caller": {"type": "direct"} the Messages API puts on each tool call, which a replay drops too; results are rewritten to Hermes' bytes, key order included), the request must extend the previous one, and a newest frame the relay cannot restore is refused. A tools/call without its id, with an id the step never received, or asked twice, for another MCP server, for a tool Hermes did not offer, or without a JSON-RPC id; a control request other than the hermes server's MCP traffic; an output line that is not a JSON object, or the end of output: the process is not trusted again, never parked or waited on, also when it comes after the step returned.
  • Version gate: kept mode runs only on the Claude Code versions the session suite passed on, listed in plugin.yaml (2.1.292, the CI pin, and 2.1.322 to 2.1.328). The version is the one each process reports in its own init event: its relay forwards nothing until that event has named an allowed version (at most 10 s). Any other version runs per-call, with one log line.
  • Memory follows the benefit: a conversation keeps a process only after session_min_rounds tool rounds. max_sessions counts every kept process of the Hermes process, across profiles. At max_sessions, the process that has waited longest for its user gives its place to the step asking now. A parked tool call is not idle. A finished subagent's process ends when Hermes closes the child's client (stock Hermes v0.23.1 does, in tools/delegate_tool_child_run.py).
  • Restart reasons at debug level: launch, edited, results, steer, exited, diverged, unproven, no_tool_id, tool_id, control, event, version; and Claude session evicted.

Settings

Read from the serving profile's config on every call (hermes config set plugins.entries.claude-subscription-directsdk-experimental.settings.<key> <value>):

Setting Default
session_mode true false: a new process for every call
session_min_rounds 8 tool rounds before a conversation keeps a process
max_sessions 4 kept processes per Hermes process
session_idle_seconds 3600 how long a kept process may wait for its user

The cost is memory between calls. During a call both modes hold one Claude Code process; a kept process holds it between calls too and grows with history. Measured here at round 200, both bench runs, at 8411b10: 273 to 292 MB on Linux, 314 to 327 MB on macOS (the review's figures: 245 to 290 MB Linux, 284 to 307 MB macOS, 325 MB the most seen). max_sessions 4 can hold about 1.1 to 1.3 GB while agents wait.

Gates

At 74a5ae5 (main 72eb3b8 merged), against hermes-agent main 95b73865, which is what CI checks out: the session test files 130 passed on each of 2.1.292, 2.1.322, 2.1.323, 2.1.324, 2.1.325, 2.1.326, 2.1.327 and 2.1.328; the full plugin suite 785 passed (and 9 subtests) and evals/ci.sh exit 0 with 73 passed on 2.1.292 (the CI pin) and 2.1.328; the bench as above. 2.1.327 and 2.1.328 passed the full suite, evals/ci.sh and the bench before they were listed (at 8631b5f and 0f35b81). On macOS (Mac mini, Apple M4, macOS 26.6.2) at 74a5ae5: the session test files with the version and concurrency tests 132 passed, and evals/ci.sh exit 0 with 73 passed, on each of 2.1.292, 2.1.322, 2.1.323 and 2.1.326; 2.1.324, 2.1.325, 2.1.327 and 2.1.328 were not run on macOS. CI at this head (https://github.com/NousResearch/hermes-official-plugins/actions/runs/44616312164): the plugin's Linux and macOS jobs pass; the Windows job fails 1 test (see Not covered).

Not covered

alt-glitch and others added 30 commits November 10, 2026 05:03
…ess (NousResearch#329)

The plugin started one claude per model call with no limit. A delegate_task
fan-out (up to 10 children by default) plus auxiliary calls started 10+
processes at the same moment, about 250 MB each, all on one subscription,
with Claude Code's own retries off, so a burst could trip the plan's rate
limits for every caller together.

Every turn now takes a place in one process-wide pool before its claude
starts (the extra-usage check included) and gives it back only once that
process has exited, on the fast-return drain thread too, so a crashed or
killed native never holds one. The pool lives under one sys.modules key
because Hermes imports the plugin once per profile home; a per-module pool
would let a multiplexed gateway run max_concurrent per profile.

A call past the limit waits in arrival order within its own timeout. It is
not failed and not sent to a fallback provider. Cancel ends the wait within
0.1 s without spawning. While waiting it yields the plugin's existing empty
liveness chunk once a second; without it Hermes' stale timer would kill the
waiting call and requeue it at the back. A wait over 2 s is logged (one
warning a minute, info after), since a model provider has no door to the
status line.

/usage, the picker handshake and auth status take no place: they send no
Messages request, end within seconds, and /usage must answer while a
fan-out holds every place.

Setting: max_concurrent in plugin.yaml config_schema, default 4, read per
call from the serving profile.
Appends from several processes to one file are not atomic on Windows; the
Windows lane lost one fake's start line (4 of 5 recorded, all 5 calls served).
…rrency

feat(directsdk): cap concurrent Claude Code processes
…iction during a park falls back to per-call bytes
…rounds, compaction, edit, park) and replay-cost bench
… texts merge into one wire message; one process per per-call step
…pped client's process ends with it

With max_sessions 1, three agents starting together kept three processes: the cap was checked under _live_lock but the
place was taken only when the launched native joined _live. A _Reservation now holds the place from the check.

The idle timer held the client's bound _end_session, so a client Hermes dropped without close() stayed referenced, and
its process with it, for the idle hour. The timer calls through a WeakMethod and a finalizer ends the process a
collected client still held. _keep refuses to publish on a client closed mid-step; settled() asks the reader thread
instead of poll(), which reaped the child behind the reader's group kill.
…vious mode's drains, budgets each turn by its own rounds
…Hermes' frames is refused before upstream and replayed
…rocess state together; a pid that vanishes mid-check counts as gone
…the parked row's pid for survival; a bare claude resolves on PATH
…ssion ends it; ownership moves under both locks

Review M2 on NousResearch#361 (and Greptile's "Cancellation misses the kept process"). The ownership-transfer change is the
reviewer's patch, kept together with the weak idle timer. The test pauses a step just before the hand-over, stops
the client from another thread and expects the cancellation error, no kept session, no process and no place held.
…equest; control writes are bounded like frames

Review M3 on NousResearch#361 (and Greptile's "Tool replies bypass the timeout"). The bound on _write is the reviewer's patch:
one timer per control write, as send() already has per frame. A write that gives up on the reader thread now reaches
the step as the timeout error too, not as bad output. The test parks a call on the real CLI, stops the child, brings
a 250,000 byte result with a 0.2 s timeout and expects the timeout error with nothing left running; the same result
on a healthy child arrives in full on the same process.
…and its place back at once

Review m1 on NousResearch#361 (and Greptile's "Dead processes consume slots"). The reader thread ends the session through the
client when native exits, and release() no longer joins the reader from the reader. A process that exits just before
it becomes the session is ended by the hand-over. Tests: kill the idle child, the place is free and a second agent
with max_sessions 1 keeps a process; kill it at the hand-over, the step returns and no place stays held.
Review m3 on NousResearch#361 (and Greptile's "Failed launches leave relays"). The launch directory is made inside the relay's
cleanup scope, as the reviewer suggested. The test spawns nothing: _launch_dir raises OSError and the relay's thread,
its listener and its place against max_sessions must all be gone.
…against max_sessions back

Follow-up to review M1 on NousResearch#361. The place was taken under the lock but released only inside _launch's own cleanup,
so a relay that could not start kept it for the life of the process. _run now gives it back however the launch ends
(the reviewer's release on every outcome), and _launch still hands the place from the reservation to native in one
step under the lock, so the count never reads one high. The test fails the relay with OSError and expects no place held.
…stream; the extension check has its own case

Review m5 on NousResearch#361. tests/test_admission_session_drift.py is the reviewer's regression, added as written in the review.
On this branch a history that does not line up with Hermes' frames is refused first, and that test's drift case has
no frames to line up with, so either refusal answers it. The new case in test_admission_session_history.py changes
the model on rows that do line up: it fails when the extension check stops refusing and passes without the
alignment refusal.
…ls/ci.sh

Review m4 on NousResearch#361 (and Greptile's "Session tests miss CI"). A runner without claude on PATH skips every test in
tests/test_directsdk_session*.py and the suite still exits 0. ci.sh already installs the pin for the evals, so it
runs those three files with CLAUDE_CODE_BIN set to it, on Linux and macOS as the evals do.
…bug level

launch, edited, results, steer, exited or diverged: resumes() answers with the reason instead of None, and one
_restart() kills, releases and logs. The divergence line moves from info to debug with the others. The session test
asserts no restart line at debug level during the conversation, and the reason for its edit of the opening turn.
…s with per-call's bytes

resumes() has a third answer for the expected results followed by one text block: the step writes the steer as a
user frame, then delivers the results. Native reads its stdin in order, so it sends one request with both, the steer as
its own system row (the same wording on 2.1.292, 2.1.322 and 2.1.323). canonical_history() puts that row back as the
text block after the results, in the newest frame and in native's history after it, and refuses a request with a steer
row it cannot match, so a Claude Code that words it otherwise restarts into per-call bytes. A steer of more than one
text block restarts at once. The real-CLI tests (one and three parked calls, both restarts) run in evals/ci.sh.
…rted and per-call

--steer 50,200 has Hermes deliver a steer after those tool rounds and adds a session_restart mode, session mode with the
steer not carried (as before), so the step's cost is reported three ways and each session mode's bytes are compared
with per-call's.
…; the steer bench and its numbers

The transport list says what a steer does on a kept process and what it costs at round 200, kept, restarted and
per-call; the steering paragraph no longer says the provider adds nothing; ci.sh's session files include _steer.py.
teknium1 and others added 29 commits November 19, 2026 11:07
…imit_event; declare and document it

The reserve only took effect on a get_usage probe, which runs only with no
current answer. After any turn the process holds the turn's own
rate_limit_event answer (using_overage false below 100%) for up to 15
minutes, so a running gateway kept sending past the reserve until the plan
wall: the setting worked only on a fresh process. The event summary now
carries the fuller account-wide window's utilization (unifiedWindows
five_hour/seven_day), and under ask/stop with permitted overage a window at
the reserve counts as spilling. The refusal says it was the reserve. The
setting is declared in plugin.yaml (Desktop settings form) and documented in
the README; CHANGELOG line under Added.
…hread per replayed frame

Every replayed frame is one stdin write, and each write started a threading.Timer
(one thread) as its stall guard: 602 thread starts for a 300-round history.
One watchdog thread per request now guards every write. A write records when it
began; the thread never sleeps past the timeout, so it wakes before the deadline
of any write that began while it slept, then sleeps until exactly that deadline.

Plugin CPU per call (4 vCPU Linux, Claude Code 2.1.327, loopback upstream):
100 rounds 0.19 -> 0.10 s, 300 rounds 0.42 -> 0.23 s. Claude Code's own CPU is
unchanged.
… at 5m, not 1h

Core clamps a delegated child to the 5m cache tier (_apply_child_cache_ttl: a
subagent calls every few seconds for minutes and is gone, so the 2x write buys
retention it never collects). Core can only clamp the cache_control markers it
builds. On this route native places them, and a `claude -p` turn is Claude
Code's main-conversation bucket, which asks for the one-hour tier on a
subscription within plan usage. Every child request therefore wrote at 1h:
2x the input rate instead of 1.25x.

The profile hook reads the cache scope core stamps on a child (sub_<hash>,
prompt_cache_scope.is_sibling_cache_scope) and returns hermes_cache_ttl=5m; the
spawn then starts native with CLAUDE_CODE_PROMPT_CACHE_TTL=5m (documented,
Claude Code 2.1.242+). A tier the user exported wins. Every other request keeps
native's default.

CLAUDE_CODE_SUBAGENT_PROMPT_CACHE_TTL is deliberately not used: it governs
requests outside the main bucket, and a -p turn is inside it.
Native's limit message names a clock time ("resets 3:45pm") that core
cannot parse, so core never knew when a used-up window reopens. The
relay now keeps the reset of a 429 whose unified status is "rejected",
and the error carries it as body["resets_at"], the field core reads for
its fallback bench and limit notice.
…ession-steer: main's fixes carried onto the kept process

NousResearch#445 lands 15 fixes on main. Where a kept process takes the same path, it gets the same fix:
close() no longer refuses or cancels a call, it ends the kept process (NousResearch#378); a delegated child's
5m cache tier is in the launch env and so in what a kept process is matched on (NousResearch#416); an incomplete
upstream answer keeps its transport phrase and plan-window reset (NousResearch#396, NousResearch#387); and one stdin watchdog
per request serves a kept step's frames too (NousResearch#419). The watchdog ends with its step, so a kept
process between steps holds neither a thread nor its client. One-shots mount no MCP server here, so
the frozen-copy test checks that a one-shot names none instead of the inert helper.
…rozen copy is deleted: plugin.yaml is read at import
… process leaves the step's answer standing; the process still ends
…ate_limit_event is read as it arrives, and a step it reports nothing on feeds the last state
…d child's 5m cache tier and for a vendor-prefixed model id
…ce, so a client dropped right after a step is collected and its kept process ends
@metahacker

Copy link
Copy Markdown
Contributor Author

Main (72eb3b8, #445) is merged in: 405aa22, a merge commit, then five fixes and one test commit. Head 74a5ae5.

Conflicts, each resolved by keeping both sides:

  • CHANGELOG.md: every entry of both sides is kept, checked line by line. Main's two max_concurrent entries and ours are now one entry that holds every sentence of each.
  • README.md: our paragraph, with main's sentence on a 429's reset added.
  • directsdk.py, five hunks: Client.__init__, close() and the hand-over, and three in _run and the step. This branch keeps its kept-process code, and main's changes are carried into it (the table below).
  • tests/test_directsdk_workspace.py: both environment lines are kept.

admission.py, concurrency.py, directsdk_setup.py, evals/ci.sh and plugin.yaml merged without a conflict. Each holds one copy of main's code, which the kept process uses as well.

Main's fixes on the kept process

Fix Kept process Test
#423 close() Gets it. close() no longer refuses or cancels a call; it ends the kept process at once. A close() that lands while a step hands its process over made that step raise "cancelled". Now the answer stands and the process ends (04ee995). main's test_client_close_parallel.py; test_a_stop_while_a_process_becomes_the_kept_session_ends_it[close] and [cancel]; test_a_dropped_client_ends_its_kept_process
#419 stdin watchdog Gets it: a kept step's frame writes use main's _StdinWatch, one per step, ended at the hand-over. A stopped watch held its request, and through it the client, until its thread exited, so a client dropped right after a step was sometimes not collected and its process lived on (9 failures in one full-suite run at a0890a3). It now lets go at once (4f9ce0e). Writing a result to a parked call keeps its own bounded write: one per result, not one per replayed frame. main's test_history_replay_starts_one_stdin_watchdog_not_a_thread_per_frame; test_a_stopped_stdin_watchdog_holds_no_request
#416 5m tier for a child Gets it: the variable is in the launch environment, which is part of what a kept process is matched on. test_a_kept_child_sends_per_calls_bytes_with_no_1h_marker: 12 requests, the same bytes as per-call, none asking for 1h
#421 vendor prefix Gets it: both paths launch with the stripped id. test_a_kept_process_on_a_vendor_prefixed_id_sends_the_plain_ids_bytes
#425 reserve from rate_limit_event Was blind. Over 12 steps, per-call fed the policy 12 times and kept 0. Claude Code writes the event only when the window changes, and a kept process often writes it after its step has ended, where the next step dropped it unread. Now it is fed as it arrives, and a step it reports nothing on feeds the last state (eb7913f), under the serving profile's settings (74a5ae5). test_every_step_feeds_the_extra_usage_policy[True] and [False]
#422 binary and --version Gets it: one binary is resolved per step and that one is started. The gate reads the version that process reports in its init event. main's test_cli_version_resolution.py; test_directsdk_session_version.py
#427, #387 incomplete answer Gets it: the same ClaudeAPIError with the transport phrase and resets_at. Only a request the relay refused before upstream is replayed. main's test_transport_phrase.py, test_directsdk_rate_limit_reset.py
#381, #382, #406 frozen copy This branch has no inert MCP helper. The version list was read from plugin.yaml on each call, so after a temp cleaner removed core's copy, every kept conversation quietly ran per-call. It is now read at import (bbe3d50). main's test_frozen_copy_removed.py, adapted: one-shots name no MCP server
#426 The workspace test keeps both environment lines. test_directsdk_workspace.py
#404, #408, #412 (ours) One copy each, used by kept steps too: the relay's close (kept processes rearm the same relay), the late slot (a kept step takes a slot per step), the bind without DNS. main's test_admission_close.py, test_concurrency_deadline.py, test_admission_bind.py; the session files

Each fix and new test fails with its change removed. The one exception is test_a_dropped_client_ends_its_kept_process, which is timing-bound; test_a_stopped_stdin_watchdog_holds_no_request checks the same fix without timing.

What ran, at 74a5ae5, against hermes-agent main 95b73865:

  • Linux: the session test files 130 passed on each of 2.1.292 and 2.1.322 to 2.1.328. The full plugin suite 785 passed (and 9 subtests) and evals/ci.sh exit 0 with 73 passed on 2.1.292 and 2.1.328.
  • The bench on 2.1.292 and 2.1.328: every kept request byte-identical to per-call's. Timings are not used: the host was not quiet.
  • macOS 26.6.2: the session test files 132 passed and evals/ci.sh exit 0 with 73 passed, on each of 2.1.292, 2.1.322, 2.1.323 and 2.1.326.
  • codex review over c234ac6..74a5ae5, in two passes. It found three P2s. One is ours and fixed (74a5ae5). Two are in the reserve code main added for feat(claude-directsdk): opt-in extra_usage reserve before the first billed request #425 and apply per-call as well, so they are not changed here: overageStatus: allowed_warning is not counted as permitted, and a cached decision outlives a change to the reserve setting.

CI at this head (https://github.com/NousResearch/hermes-official-plugins/actions/runs/44616312164): the plugin's Linux and macOS jobs pass. The Windows job fails 1 test in evals/ci.sh (1 failed, 70 passed, 2 skipped): test_a_line_the_plugin_does_not_know_restarts_the_kept_process[duplicate-toolu_00_1-tool_id]. Its first request differs from per-call's only in the device id Claude Code makes up in a fresh config directory. With #410 merged in, evals/ci.sh runs the session tests on Windows here, and nothing in this merge seeds that id. #411 holds the test-harness fix: at bb3c58b, this head with its one commit, all three jobs pass, Windows with 71 passed, 2 skipped (https://github.com/NousResearch/hermes-official-plugins/actions/runs/44617415186).

Not run: a live run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-decision Awaiting an owner policy or product decision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants