Repository navigation
fix(directsdk): stream tool calls as Claude writes them - #360
Conversation
b5bd28e to
192abf9
Compare
|
@greptileai review |
[Medium impact] No blocking defect was established; the two earlier maintainability and test-coverage comments remain non-blocking follow-ups. FindingsFix with agent prompt### Issue 1
claude-subscription-directsdk/directsdk.py:1170-1176
The new tool state is split between `Capture.arguments`/`unparsed_calls`, `_ToolStream.raw`/`slots`, and overlapping `calls`, `parsed_calls`, and `unparsed` lists here. Finalization then reaches into `tool_stream.slots` to reconcile them. Changing tool-input rules now requires matching changes across these representations.
This violates the repository requirement to keep feature logic behind a dedicated abstraction rather than scatter special cases. Consolidate these rules before merging. Use one ordered per-tool model with raw input, parsed input, and streamed position. Derive final calls, replay data, and remaining deltas from it. Keep the two input streams separate, but share their tool-state rules.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
### Issue 2
claude-subscription-directsdk/tests/test_directsdk_tool_stream.py:136-140
This test checks that the plugin emits parseable arguments and `content_filter`, but `len(bodies) == 1` only counts the single `Client.create` call. Hermes decides whether to retry after assembling those chunks, so this test can pass without protecting that behavior.
Feed the actual chunks through the existing `_hermes_stream_response` and `_hermes_reply` helpers. Check that Hermes drops the tool and shows the refusal. Include malformed eager input too: `_json_suffix` returns `None` for `{"path": /tmp/a.txt}`, so that case follows a different path from a cut-off valid string.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
|
| parsed_calls = list(calls) | ||
| # A call whose arguments did not parse (cut at max_tokens, or invalid streamed JSON) reaches Hermes with | ||
| # its raw arguments and finish length: Hermes never runs a call from a length reply, it re-sends the | ||
| # request with a larger output cap (#164, #221). A refused call stays out (below). | ||
| unparsed = [] if refused or not admission.used else admission.capture.unparsed_calls | ||
| for call_id, native_name, raw in unparsed: | ||
| calls.append({'id': call_id, 'type': 'function', 'function': {'name': offered_tool_name(native_name or '', hermes_names)[1], 'arguments': raw}}) |
There was a problem hiding this comment.
The new tool state is split between Capture.arguments/unparsed_calls, _ToolStream.raw/slots, and overlapping calls, parsed_calls, and unparsed lists here. Finalization then reaches into tool_stream.slots to reconcile them. Changing tool-input rules now requires matching changes across these representations.
This violates the repository requirement to keep feature logic behind a dedicated abstraction rather than scatter special cases. Consolidate these rules before merging. Use one ordered per-tool model with raw input, parsed input, and streamed position. Derive final calls, replay data, and remaining deltas from it. Keep the two input streams separate, but share their tool-state rules.
Context Used: Perform a deep code quality audit of the current branch's changes. Rethink how to structure / implement the changes to meaningfully improve code quality without impacting behavior. Work to improve abstractions, modularity, reduce Spaghetti code, impr... (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: claude-subscription-directsdk/directsdk.py
Line: 1170-1176
Comment:
**Tool state has several owners**
The new tool state is split between `Capture.arguments`/`unparsed_calls`, `_ToolStream.raw`/`slots`, and overlapping `calls`, `parsed_calls`, and `unparsed` lists here. Finalization then reaches into `tool_stream.slots` to reconcile them. Changing tool-input rules now requires matching changes across these representations.
This violates the repository requirement to keep feature logic behind a dedicated abstraction rather than scatter special cases. Consolidate these rules before merging. Use one ordered per-tool model with raw input, parsed input, and streamed position. Derive final calls, replay data, and remaining deltas from it. Keep the two input streams separate, but share their tool-state rules.
**Context Used:** Perform a deep code quality audit of the current branch's changes. Rethink how to structure / implement the changes to meaningfully improve code quality without impacting behavior. Work to improve abstractions, modularity, reduce Spaghetti code, impr... ([source](https://app.greptile.com/custom-context?memory=instruction-0))
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| with upstream(tmp_path, reply(['{"path": "/tm'], 'refusal', details)) as (client, bodies): | ||
| chunks = list(client.create(**REQUEST, stream=True)) | ||
| arguments = ''.join(d.function.arguments or '' for d in tool_deltas(chunks)) | ||
| assert json.loads(arguments) == {'path': '/tm'} | ||
| assert chunks[-1].choices[0].finish_reason == 'content_filter' and len(bodies) == 1 |
There was a problem hiding this comment.
This test checks that the plugin emits parseable arguments and content_filter, but len(bodies) == 1 only counts the single Client.create call. Hermes decides whether to retry after assembling those chunks, so this test can pass without protecting that behavior.
Feed the actual chunks through the existing _hermes_stream_response and _hermes_reply helpers. Check that Hermes drops the tool and shows the refusal. Include malformed eager input too: _json_suffix returns None for {"path": /tmp/a.txt}, so that case follows a different path from a cut-off valid string.
Prompt To Fix With AI
This is a comment left during a code review.
Path: claude-subscription-directsdk/tests/test_directsdk_tool_stream.py
Line: 143-147
Comment:
**Refusal tests stop too early**
This test checks that the plugin emits parseable arguments and `content_filter`, but `len(bodies) == 1` only counts the single `Client.create` call. Hermes decides whether to retry after assembling those chunks, so this test can pass without protecting that behavior.
Feed the actual chunks through the existing `_hermes_stream_response` and `_hermes_reply` helpers. Check that Hermes drops the tool and shows the refusal. Include malformed eager input too: `_json_suffix` returns `None` for `{"path": /tmp/a.txt}`, so that case follows a different path from a cut-off valid string.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.…message Claude Code 2.1.292 and 2.1.319 send today's date as "<system-reminder>\nToday's date is ...\n</system-reminder>", but _DATE_LINE only matched the bare 2.1.299 text. The date-move branch never fired, so the environment message (messages[1]) carried the date on the opening request and lost it from request 2 on, leaving a trailing role:system date row. That is the residual of #38: an edit of an early message before the turn's thinking. The regex now accepts the optional wrapper. The unit fixtures use the wire text, and the cache eval asserts messages[1] reads the same on every request of a fresh conversation (red on the base: "native environment message changed after the opening request"; green on 2.1.319 and 2.1.292).
Salvaged from P#21 (cherry-pick of f4faf14, rebased onto main: the import hunk conflicted because main already imports hashlib). Closes P#21.
…egment) to the offered tool Native Claude Code sometimes names a tool mcp__<tool>, dropping only the server segment of mcp__hermes__<tool>. directsdk handled the full prefix and the bare name (#62) but not this form, so core's _repair_tool_call fixed it and logged "Auto-repaired tool name" on every such call (cosmetic, no failures). Map it back to the offered name in the tool_use loop, which also keeps the replayed history on the offered name. Test: test_server_segment_slip_maps_to_the_tool_and_replays_with_the_offered_name (fails without the fix). Merged from P#342 as is (cherry-pick of b05ffb8). Closes P#342.
… not retried (#327 #328) Salvaged from P#336 (kanederkson, ebcff9c only; its advanced-delegation and mcp-serve commits are unrelated and stay out). unsupported_response_formats before the first auxiliary request. The plugin refuses schema-less json_object, so declare it. Set by attribute after construction, as supports_streaming is: a constructor kwarg fails on a host that predates the field (requires_hermes >= 0.21.4). said failure_retryable True. Every pre-spawn refusal in directsdk.py now raises RequestRejected (a ValueError, so existing callers are unchanged), and the profile hook maps it, by class name, to format_error with retryable False and should_fallback True, so a different fallback provider still gets its try. A plain ValueError from anywhere else keeps core's own verdict, unlike P#336's blanket ValueError/TypeError match. Tests red on the base: the refusal verdict and the auxiliary drop. Closes P#336, #327, #328.
…th on the per-call path Native keeps a call whose streamed arguments are not valid JSON and answers it locally with a parse error, so --max-turns 1 ends the turn with error_max_turns and exit code 1. The relay drops the unparsable call at message_stop, so the answer carried no calls and the boundary judged the turn a failure: the step raised RuntimeError Native request failed: error_max_turns instead of returning the text it already had. Treat a turn whose calls were all dropped (admission.capture.unparsed) like a turn with calls at the error_max_turns boundary, also in _fast_eligible. The answer reaches Hermes with finish length and no tool calls, as 55c1b45 described for the relay path (#164). Salvaged from P#171 (cherry-pick of e15ee45; its CHANGELOG hunk conflicted and is dropped, the next commit writes the entry). The next commit changes the outcome: the cut call reaches Hermes with its raw arguments instead of being dropped.
…ken one (#164 #221 #326) One root cause behind three reports. The relay dropped a tool call whose streamed arguments did not parse, and the plugin sent no tool-call delta before the final chunk. - A cut (max_tokens) or invalid call now reaches Hermes with its raw arguments and finish length. Core never runs a call from a length reply; it re-sends the request. Finish tool_calls would be unsafe: core repairs and runs an unclosed argument. Previously #221 failed the turn after 3 attempts (error_max_turns) and a mixed reply silently lost the bad call. - eager_input_streaming stays on by default (turning it off regressed the #156 stall fix: a 6 s zero-event gap live). After an invalid (not cut) reply, that conversation's next requests buffer tool input so the API validates it, the same knob core's Anthropic transport turns on. - The tool call header (id, name) streams at content_block_start and each argument fragment streams as a chat-completions tool_calls delta, so Hermes shows "Preparing <tool>" at once. A refusal mid-call gets a closing suffix that makes the streamed arguments parse, so core's accumulator keeps content_filter (one request) instead of turning it into a truncation it retries (the triage prototype measured 5 requests). - One name mapping (offered_tool_name) serves the header stream and the final calls, so P#342's server-segment slip applies to both. P#171's fast-path clause and tests are kept from the previous commit; its tests now assert the cut call reaches Hermes with raw arguments. Through core's real turn loop (e2e_221.py, real CLI, fake upstream): invalid JSON done in 2.6 s (was: failed after 3 attempts, 11.98 s); cut done in 2.6 s with no error; unclosed done; refusal mid-call and after a parsed call both end content_policy_blocked after 1 request, 0 tools run. Tests red on the base: tests/test_directsdk_tool_stream.py (5). Admission eval gains a bad_tool mode. Not done here: reporting native's output cap through get_max_tokens. It would put max_tokens on every request (CLAUDE_CODE_EXTRA_BODY and CLAUDE_CODE_MAX_OUTPUT_TOKENS), and core's boost would then ask for 2x the model cap on a chat_completions route, where _model_output_limit is None. No clean seam; left as a core follow-up. Closes #164, #221, #326. Supersedes P#256 (eager off by default, adds a non-secret env var, does not fix the cut) and P#338 (removes eager streaming, bundles 4 unrelated commits).
… in message.refusal (#341) An upstream refusal left no trace: the request id appeared in no log, so a user could not ask Anthropic support about it. Each refusal now logs one WARNING with model, request_id and stop_details.category, and message['refusal'] ends with (request_id=req_...). The id passes the same bounded req_ check ClaudeAPIError already applies, now one helper. Core still prefers preamble content over provider_data['refusal'] when text came first (agent/turn_truncation.py handle_content_policy_refusal); that is a core follow-up. The log line covers that case meanwhile. Test red on the base: test_refusal_names_its_request_id_in_the_log_and_the_message. Closes #341.
Merges the invalid-JSON and cut-at-max-tokens cases, which drive the same path (the call is returned with finish length, never run, and the next request picks how the API streams the arguments), into one parametrized test. Drops the exhaustive _json_suffix prefix test: the refusal-mid-call test already proves the streamed arguments of a cut call parse. Each fix commit's code, reverted alone, still fails a kept test.
192abf9 to
68f030a
Compare
|
@greptileai review |
unsupportedpastels
left a comment
There was a problem hiding this comment.
Review: PR #360 fix(directsdk): stream tool calls as Claude writes them @ 68f030a8
Verdict: 🟡 Changes requested. The normal tool-streaming and malformed-input recovery paths work, but a refusal after streamed tool input no longer reliably reaches Hermes as a terminal refusal. There is one integration issue to address without turning streaming off.
Direction: This follows #164, #221, #326, #327, #328 and #341: expose tool preparation, recover broken calls without executing them, declare already-unsupported JSON-object mode, avoid deterministic retries, and retain refusal diagnostics. The four salvaged contributions retain their authorship. The refusal behavior requested in #326 is not yet preserved through core's stream assembly; see M1.
| Critical | Major | Minor | CI |
|---|---|---|---|
| 0 | 1 | 0 | — none |
Checked:
- Compared the complete 12-file delta against #357 at
a8b0fa89; traced provider registration →Client.create→ request validation/history → relay capture → native stream events → final chunks → Hermes' actual_StreamingCall._call_chat_completionsand response normalizer. Also read the plugin's AGENTS, README, CHANGELOG and repository contribution rules. - Linux/Python 3.14: full head suite 5 failed, 576 passed, 9 subtests passed; #357 suite 5 failed, 557 passed, 9 subtests passed. All five failures are inherited, not findings here. The four refusal-helper failures need main's #370 test-stub update; the other failure is the Linux
no_uidsimulation under the stripped test environment and passes withUSERrestored. - All seven changed test files: 103 passed,
RUNNER_RC=0. Copied only those tests onto #357: 23 failed, 80 passed; the failures exercise date folding, names, ID normalization, broken calls, streaming and diagnostics. The pre-start rejection test's missing exception class is not, by itself, behavioral proof. - Default eager streaming stays on; complete tools and no-argument tools still return one call in one upstream request. Broken nonrefused arguments and truncated arguments finish
length; only invalid, nontruncated JSON disables eager input on the next request. Foreign IDs stay paired, valid IDs and signed replay stay intact, and offered-name mapping preserves the original tool name. Existing async, cancellation, workspace, credential-isolation and proxy/CA tests ran in the full suite. - Both offline native evals passed on Claude Code 2.1.292, including the 25-request cache-wire check. A paced real-native positive control emitted the tool header at 1.111 s, before the final chunk at 2.311 s, with argument fragments assembled exactly once; the fix retains the same three tool deltas. Eleven isolated production mutations each made a PR test fail. Two concurrent fixture calls completed independently; eight threads inserting 1,000 conversation flags retained the configured 256 entries.
- Trial merge with current main
8b5b6296: the only textual conflict is the inherited plugin version choice, already present for #357. Keeping0.4.0gives 1 failed, 584 passed, 9 subtests passed; only the sameno_uidharness case remains. Notestsworkflow ran at this stacked head; the CI cell does not represent the local suite.
Not checked: Live model inference, the private timing/cache/billing receipts or the two screenshot claims, Claude Code 2.1.319, and Windows/macOS execution. No platform-specific finding is inferred from Linux results.
Security: Checked the changed native-control-protocol and history inputs against the plugin's credential/host-execution promises and Hermes SECURITY.md §§2.2–2.6: argument injection, profile credential scoping, child environment, loopback routing, temp-directory/symlink handling and process cleanup. No new security-boundary issue found; M1 is a refusal/retry correctness issue, not containment against an adversarial model.
Thanks @alt-glitch! Keeping eager streaming enabled while handling malformed input separately is worth preserving.
Critical — none
Major
M1: A refusal after streamed tool input loses its explanation, or becomes a retryable truncation · directsdk.py / core stream assembly
-
Where: tool fragments emitted before completion, suffix-only refusal finalization, and the test that stops before Hermes assembly.
-
Problem: When native has already streamed a tool header and argument bytes before a no-text refusal, clearing the final response object's calls cannot retract the earlier deltas. Hermes assembles those deltas rather than adopting
chunk._response. With the repairable prefix{"path":"/tm, the tool remains in the assembled message, suppressing promotion ofmessage.refusalinto displayed content. With malformed eager input{"path": /tmp/a.txt},_json_suffixcannot repair it, and core changescontent_filtertolength. The latter enters truncation recovery instead of the terminal refusal path. #357 returned neither call and preserved the reason. This confirms and extends @greptile-apps' “Refusal tests stop too early” observation. -
Evidence: Real Claude Code 2.1.292, a paced loopback SSE peer with named events, and Hermes' actual stream driver; no model inference or real login. Each row is a separate request.
TREEselects the reviewed checkout;CLAUDEselects the pinned executable.$ python -I real_driver.py "$TREE" --claude "$CLAUDE" --check {"raw": "{\"path\":\"/tm", "finish": "content_filter", "tools": 1, "content": null, "requests": 1} {"raw": "{\"path\": /tmp/a.txt}", "finish": "length", "tools": 1, "content": null, "requests": 1} REFUSAL_FAILURES=2 PROBE_RC=1The identical probe on #357 reports
content_filter, zero tools andFlaggedfor both rows. With the companion fix below, both reportcontent_filter, zero tools andFlagged (request_id=req_fixture01);REFUSAL_FAILURES=0, exit 0. This demonstrates the wrong finish/reason, not an actual repeated paid request or tool execution. -
Fix: I'd suggest pairing this with a core change that discards accumulated tool fragments on a provider-confirmed
content_filterbefore argument repair and refusal promotion, then requiring a core version containing that change. A three-line companion fix was verified on a disposable core copy; all 103 PR changed-file tests remain green without edits. Add a regression that feeds the real streamed chunks into core for both prefixes above; checking only the last chunk and oneClient.createmisses the problem.Verified companion core change
--- a/agent/chat_completion_helpers.py +++ b/agent/chat_completion_helpers.py @@ + # A terminal provider refusal invalidates every tool fragment already streamed. + if finish_reason == "content_filter": + tool_calls_acc = {} mock_tool_calls, has_truncated_tool_args = self._assemble_tool_calls(tool_calls_acc, finish_reason)
-
Feature check: Ordinary complete tools and no-argument tools still complete in the default streaming configuration at #357, head and fix; malformed nonrefused calls still finish
lengthat head/fix. The fix does not disable tool headers, argument streaming, signed replay or eager input, and all 103 changed-file tests pass with it.
Minor — none
Good
- Foreign-ID normalization reserves already-valid identifiers first and rewrites calls/results together without altering source history or payload strings.
- Broken JSON retains its raw arguments and changes the next request's eager-input policy only when the upstream stop was not
max_tokens. - Refusal request IDs are bounded and validated before entering diagnostics, rather than logging arbitrary header contents.
|
M1 is valid. Tip: You can customize Greptile's behavior for this repo with |
|
@greptileai review |
This comment has been minimized.
This comment has been minimized.
|
@greptileai review |
Merges after #357; stacked on it.
Tool calls now reach Hermes as Claude writes them, so a long
write_fileshows "Preparing write_file…" instead of a bare spinner, and a broken or cut-off tool call is retried instead of failing the turn. Closes #164, #221, #326, #327, #328, #341.Before
On #357's head, the relay held each tool call until Claude finished writing it:
write_fileshowed only the thinking spinner until the whole call was written: 30 s for a 1,500-word file.max_tokens, was dropped. Claude Code then failed the turn witherror_max_turns, and each retry failed the same way.After
The relay passes each tool call on as Claude streams it:
lengthand its raw arguments. Hermes never runs a call from alengthreply; it retries.What the user sees
write_filecallmax_tokens(#164)Native request failed: error_max_turns(request_id=req_...)for Anthropic's false-positive form (#341)response_format: json_objecton an auxiliary call (#327)ValueErroron every callterminal:42)toolu_hermes_<hash>id, paired with its resultmcp__<tool>The interactive CLI on the same prompt. The after row reads "wriie_file" because the capture's tmux counts the width of the ✍️ emoji differently from the CLI; in a terminal it reads "write_file".
Not in this PR
lengthretry does not shrinkmax_tokens: no plugin seam yet.Live verification
hermes -p <scratch>, "write a 1,500-word guide in one write_file call", real Claude Code 2.1.319, Sonnet 5.5lengthwith 1 upstream request; a refusal answerscontent_filterwith 1 upstream requestBoth shots are the same moment, 15 s after the prompt was sent. The two runs wrote files of different length (37.7 s and 53.5 s turns), so only the 15 s frame is a like-for-like comparison.
Cherry-picked with authorship: #21 (@efron), #342 (@mwe-ai), #336 (@kanederkson, the relevant commit), #171 (@salch-cred, the
_fast_eligiblehunk and its tests). Superseded: #256, #338 (both turned eager streaming off, which brought back multi-second silent gaps).Follow-ups
Next PR
#361 (one Claude Code process per Hermes agent) is stacked on this branch and moves here when this merges.