Repository navigation
fix(directsdk): telemetry off, full /usage, own login per profile - #357
Conversation
The CLI skips its get_usage fetch under CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC (rate_limits: null on 2.1.301/2.1.302), so a quiet-mode user lost /usage entirely. The usage probe is zero-inference and runs behind the unused admission gate; it now drops only that one flag, keeping DISABLE_TELEMETRY and DISABLE_ERROR_REPORTING. A flag the user exported in their own environment is kept and still named in the reason.
…lete Claude Code telemetry and error reporting are now opt-in (#331): the claude_code_telemetry default is false in both directsdk_setup and the plugin.yaml config_schema, so the settings UI and the code agree. Two zero-inference probes lift exactly the flags that would blind them, unless the user exported one: - the picker's initialize handshake runs without DISABLE_TELEMETRY and CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC. Either one stops the CLI's feature-flag fetch (2.1.319: 13 picker rows become 5). Keeping them unset for the handshake gives the same 12 routes as telemetry on, measured live; pinning the four missing models instead would freeze their labels and miss the next flag-gated model. - the get_usage probe runs without the nonessential-traffic flag (P#258). usage_windows now always appends the extra-usage state, credits spent, the weekly per-surface breakdown and model_scoped rows. They were dropped whenever the answer carried `limits`, which 2.1.319 always does. Supersedes P#337 (credit @salch-cred for the default flip); builds on the cherry-picked P#258 (@ephyria-k).
…tiplexing (#313) A multiplexed gateway never loads a served profile's .env into os.environ, so its CLAUDE_CODE_OAUTH_TOKEN never reached Claude Code, and an inherited token overrode the per-profile claude_config_dir login. Resolve the token through the secret scope, set it on the child env only, and drop an inherited token when claude_config_dir selects the login. Single-profile installs and Client(env=...) are unchanged.
…login P#314 (cherry-picked above, @kalfian) passes the serving profile's CLAUDE_CODE_OAUTH_TOKEN to the child, but it still let the launch profile's token through in two cases, which is the #313 leak again: - a served profile with no token and no claude_config_dir kept the inherited token; - every exception from get_secret, UnscopedSecretError included, was swallowed and the inherited token stayed on the child. Now, when the task serves a profile other than the process's own (agent.secret_scope.serves_routed_profile), an inherited CLAUDE_CODE_OAUTH_TOKEN, ANTHROPIC_CUSTOM_HEADERS and CLAUDE_CONFIG_DIR are dropped unless the profile's own scope supplies them. Only ImportError (a core without the scope API) is tolerated; UnscopedSecretError propagates from the turn, as get_secret_str's contract requires, and the setup probes degrade without spawning. Red on P#314 as cherry-picked: 9 of 18 credential tests fail. Closes #313. Salvages P#314.
Claude Code emits a rate_limit_event carrying the subscription window state (rateLimitType, overageStatus, isUsingOverage); the receive loop discarded it, leaving quota policy with no signal. Summarize it secret-free into native_admission.rate_limit. Paid use is flagged only when actually in use: overageStatus allowed alone means permitted, and the subscription windows (five_hour, seven_day*, seven_day_overage_included) are not paid use. Refs #43.
Claude Code reports on every request whether the account has passed its plan limit and is drawing paid extra usage (rate_limit_event isUsingOverage, resetsAt). P#281 (cherry-picked above, @salch-cred) stopped dropping that event; this acts on it per profile (#43): - allow: unchanged. notify (default): one warning per limit window. - ask: refuse before spawn until extra_usage_allowed_until covers the window; the message prints the exact `hermes config set` line. - stop: refuse until the plan limit resets; the message names the reset. State lives in memory per process, keyed by the child's login (config dir plus a hash of an OAuth token); nothing is written to disk. With no current answer under ask/stop, one zero-inference get_usage decides (extra usage on and the session or weekly limit at 100%). An unknown mode counts as stop. The refusal (ClaudeExtraUsageBlocked) classifies as billing with error_context.usage_window, so core shows the message and never switches provider on a plan limit. The "Third-party apps now draw from your extra usage" 400 (P#296) maps to the same billing stop instead of an unknown error that core answered silently from a fallback provider. A provider plugin cannot register a slash command (model-provider manifests skip register(ctx)), so ask is allowed via a config line. directsdk.py: only the child-env section (pre-spawn check) and the rate_limit_event branch P#281 added are touched. Supersedes P#282 (@creditlabsonarc-rgb): no marker file in the CLI's config dir, and the reset time comes from native instead of a guess.
Organizations that route model traffic through their own LLM gateway, an egress CONNECT proxy or a TLS-inspecting proxy had no supported way to do it: the relay always dialled https://api.anthropic.com and the ANTHROPIC_BASE_URL guard (correctly) refuses an inherited endpoint (#46). Three config_schema settings, read per profile and per spawn: - upstream_url: https:// only, no userinfo/query/fragment; - upstream_proxy: http://host:port CONNECT; its user:password is a secret in CLAUDE_SUBSCRIPTION_DIRECTSDK_PROXY_AUTH (profile .env, read through the secret scope); - upstream_ca_file: PEM added to the system roots; CERT_REQUIRED and hostname checks stay on. Any one set is the opt-in, and each one set must validate or the turn, /usage, setup and discovery refuse with ClaudeUpstreamConfigError naming the setting (classified non-retryable, no fallback). Nothing falls back to the direct route. Every relay uses the resolved upstream: turns, fetch_usage (was hard-coded) and the picker handshake, which goes through the relay under an opt-in. ANTHROPIC_BASE_URL stays refused. One warning per profile and route names the host the bearer goes to, with "TLS inspected" when a CA file is set. directsdk.py: only the relay construction in the child-env section. admission.py: Admission takes an explicit proxy and TLS context. Supersedes P#257 (@salch-cred: an env var selected the upstream, and a bad rebase deleted fetch_usage), P#130 (@alextidalrise: env-based CA and proxy, broad except) and P#23 (@MematiBas42: env-only proxy). Closes #46.
_install_prefixes joins HOME with .local/bin and the other home prefixes. When HOME is relative, those paths resolve against the current working directory, so _resolve can pick a `claude` binary placed in the directory Hermes runs from. Only an absolute HOME (or USERPROFILE) now contributes home prefixes. The system prefixes and PATH lookup are unchanged. Validated: new test fails on main (relative-home case) and passes on this branch; full suite 210 passed, 9 subtests passed.
`claude auth status` reports loggedIn: true for every way native can authenticate, including a Console API key (authMethod api_key or api_key_helper) and a cloud backend (authMethod third_party, apiProvider bedrock/vertex/...). setup_status passed all of them as a working subscription login. A Console key stored by the CLI is not caught by the request-path environment guard, so requests would bill API usage instead of the subscription. setup_status now treats those logins as not logged in, with a detail that says why and how to switch. On a TTY the setup gate then runs the CLI's own `claude auth login`. Model discovery degrades to the pinned table as it does for a logged-out CLI. claude.ai and oauth_token (setup-token) logins are unchanged; oauth_token reports no subscriptionType and stays valid. Checked against Claude Code 2.1.286 `auth status`: ANTHROPIC_API_KEY -> api_key/firstParty, CLAUDE_CODE_USE_BEDROCK -> third_party/bedrock, CLAUDE_CODE_OAUTH_TOKEN -> oauth_token/firstParty, claude.ai login -> claude.ai/firstParty/max.
Salvage of P#28 (cherry-picked above, @100yenadmin). Plugin 0.3.1 drops ANTHROPIC_API_KEY from every child, so "unset ANTHROPIC_API_KEY" no longer helps anyone; the hint now names the Bedrock/Vertex/Foundry switches, which still reach `auth status`. The authMethod/apiProvider gate stays as is. Adds the 2.1.319 Bedrock shape (third_party/bedrock) measured in triage (setup_checks.txt) to the test. CHANGELOG entries for P#27 (@grantjayy, cherry-picked as is) and P#28.
Picker and /usage probes (C#146288). `hermes status` refreshes the model catalog on a daemon thread; when the interpreter exited while that thread was inside the picker handshake, the TemporaryDirectory finalizer ran with claude.exe still using it as cwd: a WinError 32 traceback, a leaked dir, and a claude.exe alive for ~3 s (Windows report, finding 1). ignore_cleanup_errors alone would only hide it. The probes now spawn with Popen in their own process group, kill and wait for the child (tree) before the scratch dir goes on every path, and an atexit reaper registered after the first TemporaryDirectory (atexit is LIFO) kills a probe still running at exit before tempfile's finalizers. ignore_cleanup_errors=True stays as the last guard. Refresh lock (salvaged from P#25, the reap only). A Claude Code that exits mid-refresh leaves `.oauth_refresh.lock`, which it never reclaims (anthropics/claude-code#95236), and every later request fails. Before every probe and turn spawn, an empty lock in the child's resolved config dir older than Claude Code's 60 s stale window is rmdir'd; a live holder bumps it every 5 s and rmdir refuses one with contents. Not taken from P#25: the `claude -p /usage` settle on every setup probe (about 0.6 s and one account request per picker load) and any read of `.credentials.json`. directsdk.py: one reap call in the child-env section. Co-authored-by: levsky22 <56281588+LevSky22@users.noreply.github.com> Co-authored-by: 100yenadmin <100yenadmin@users.noreply.github.com>
New settings (extra_usage, upstream_*), a changed default (claude_code_telemetry off) and the per-profile login fix need a version hermes plugins list can show.
|
@greptileai review |
Tested 6 flows, found no issues. What we tested
The pull request merged before TREX finished testing |
[High impact] The PR appears safe to merge; both numbered findings are addressed and no new blocking issue was established.
|
This comment has been minimized.
This comment has been minimized.
BearHuddleston
left a comment
There was a problem hiding this comment.
TL;DR: No blockers from me on this head. There are 7 nits: one inline from me, and the rest are my notes on Greptile's threads, three of them with tested patches below. I don't agree that Greptile's P1s block merge: after reproducing each one, I rate two as nits, one as a nit-level design question, and one as working as designed.
| # | Finding | Severity | Where | Status |
|---|---|---|---|---|
| 1 | A probe whose launcher exits leaves the launcher's child running (Greptile) | Nit | directsdk_setup.py:277 |
Open, patch A |
| 2 | apply_profile_credential raises TypeError when routed with no scope bound (Greptile) |
Nit | directsdk_setup.py:209 |
Open, patch C |
| 3 | A non-text upstream_* value reads as unset instead of refusing (Greptile) |
Nit | directsdk_setup.py:574 |
Open, patch B |
| 4 | Legacy fractional utilization: 1.0 shows as 100% but doesn't count as full for ask/stop (Greptile) |
Nit | extra_usage.py:107 |
Open |
| 5 | A non-list rows/model_scoped makes usage_windows raise (Greptile) |
Nit | __init__.py:81 |
Open |
| 6 | upstream_url accepts loopback http://, but the message and docs say https only |
Nit | directsdk_setup.py:587 |
Open, inline |
| 7 | Plugin AGENTS.md still describes the pre-PR picker and bearer-forwarding rules |
Nit | AGENTS.md:29 |
Open |
Reproductions, against hermes-agent origin/main:
- 1:
test_a_probe_whose_launcher_exited_leaves_no_childbelow fails on this head ("the launcher's child outlived the probe"):_kill_treereturns early onpoll(). Patch A has_kill_treereusedirectsdk.kill_process_tree, whose_unreapedownership check is meant for this case.conftest'sFakePopen.pidgoes from 0 toos.getpid(), becausewaitid(P_PID, 0)is EINVAL. Rare in practice, sinceclaudeis usually a single process. - 2:
set_hermes_home_override(<other home>)with no scope bound:apply_profile_credential,setup_statusandfetch_usageall raiseTypeError: 'NoneType' object is not callable. Nothing is spawned, so it stays closed. Getting here takes a half-bound path, because core'sserveturns multiplex on for the first non-launch home. Patch C drops the inheritedCLAUDE_CONFIG_DIRthe way the loop above already drops the token, rather than raising: the(None, True)branch is designed to drop, not raise. - 3:
upstream_proxy: 3128(whichhermes config setstores as an int) gives the direct route withopted_in=False. I rate it a nit: only a value invalid for any intent gets here, core's plugin loader already warns on the schema type, and the fallback is the defaultapi.anthropic.com. Afalsemeant as "disable" would refuse under patch B, with a message that names the key. - 4:
observe_usage(..., {"extra_usage": {"is_enabled": true}, "five_hour": {"utilization": 1.0}})records not-using, whileusage_windowsshows the same window as 100%. Responses withlimits(2.1.319) aren't affected, and the nextrate_limit_eventstill catches it. Normalizing with_percent(..., fraction=True)before the>= 100check fixes it. - 5:
usage_windows({"limits": [...], "seven_day_breakdown": {"rows": 5}})and{"model_scoped": 7}both raiseTypeError: 'int' object is not iterable, andfetch_account_usagecalls it without atry. Checking that both values are lists before iterating fixes it. - 7 (
AGENTS.md, not in the diff): "Theinitializehandshake therefore runs without the relay unless the environment carries an auth/backend override" should add "or the profile opts into anupstream_*gateway (the picker is then the CLI's generic list)". The Contributor PRs line calls a knob that forwards the subscription bearer to a third-party host "a policy change, not a feature".upstream_urlis that knob, so the line should name the opt-in and its fail-closed rules. That is Teknium's call as merge arbiter, and my approval covers the code, not that policy decision.
On Greptile's other threads:
- Profiles inherit the default login: working as designed, not a regression. The
~/.claudethat remains is the OS user's default login, which the same profile run standalone (hermes -p <name>) uses too. #313 was the launch profile's.envtoken, which is now dropped (the base inherited it outright). - Failed checks allow paid turns: deliberate ("do not refuse on a guess"). That turn's
rate_limit_eventrecords the overage and the next turn is refused, which is the same one request the README already says is billed. Refusing on an unknown answer would block every turn wheneverget_usagetimes out. One limit is worth a README sentence: the bound depends on that event arriving, because the cached "not using" answer otherwise lasts 15 minutes. - Gateway probes lack traffic checks is a coverage limit. A wire-level check would have to keep the probes zero-inference.
- Login preparation disagrees across paths applies only to an explicit
Client(env=...), which is fixture-only today.
Thanks for the PR. On this head, the claude-subscription-directsdk CI lane passed on ubuntu, macOS and Windows. Locally against hermes-agent origin/main, the plugin suite gave 574 passed, and 579 with patches A–C and the inline suggestion applied. Each patch applies alone to 980bdbb with git apply, and its new test fails on this head. The PR's own tests fail on the base (85e835c production code plus head tests: 49 failed and 8 errors in the files the PR body lists). For the credential tests the meaningful red is on P#314 as cherry-picked (0e18446): 9 failed / 9 passed, as the PR body says. Claude Code's env-var docs confirm two premises: DISABLE_TELEMETRY and CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC each skip the feature-flag fetch, and CLAUDE_CODE_OAUTH_TOKEN takes precedence over the stored login. I didn't run a live turn, the picker or /usage against a real account, the Windows probe-reaper path, or a real isUsingOverage event. A GPT Sol loop (3 rounds) agreed with these severities.
Patch A: probe cleanup reuses kill_process_tree (#1)
diff --git a/claude-subscription-directsdk/conftest.py b/claude-subscription-directsdk/conftest.py
index 5cb59af..b77c662 100644
--- a/claude-subscription-directsdk/conftest.py
+++ b/claude-subscription-directsdk/conftest.py
@@ -53,7 +53,8 @@ def popen_from_run(run, turns=None):
the CLI's probes (the plugin spawns its zero-inference probes with Popen so it can own the child). A turn
spawn (``--max-turns`` in argv) goes to ``turns``, the real Popen, when given."""
class FakePopen:
- pid, returncode = 0, 0
+ # Never one of our children, so the probe's kill_process_tree sees nothing it owns (pid 0 is EINVAL to waitid).
+ pid, returncode = os.getpid(), 0
def __new__(cls, argv, **kwargs):
if turns is not None and "--max-turns" in argv:
diff --git a/claude-subscription-directsdk/directsdk_setup.py b/claude-subscription-directsdk/directsdk_setup.py
index 5e8667e..78c28e4 100644
--- a/claude-subscription-directsdk/directsdk_setup.py
+++ b/claude-subscription-directsdk/directsdk_setup.py
@@ -16,7 +16,6 @@ import json
import logging
import os
import shutil
-import signal
import ssl
import subprocess
import tempfile
@@ -275,17 +274,12 @@ _reaper_registered = False
def _kill_tree(process):
- """Kill a probe child and what it started (the npm shim is cmd.exe -> node on Windows)."""
- if process.poll() is not None:
- return
- if os.name == "nt":
- subprocess.run(["taskkill", "/F", "/T", "/PID", str(process.pid)], stdout=subprocess.DEVNULL,
- stderr=subprocess.DEVNULL, check=False)
- else:
- try:
- os.killpg(process.pid, signal.SIGKILL) # windows-footgun: ok - the nt branch above never reaches this line
- except (ProcessLookupError, PermissionError):
- pass
+ """Kill a probe child and what it started, even after the launcher exited (the turn path's helper)."""
+ try:
+ from .directsdk import kill_process_tree
+ except ImportError:
+ from directsdk import kill_process_tree
+ kill_process_tree(process)
def _reap_probes():
diff --git a/claude-subscription-directsdk/tests/test_directsdk_probe_lifecycle.py b/claude-subscription-directsdk/tests/test_directsdk_probe_lifecycle.py
index 6fdf854..62ccb6d 100644
--- a/claude-subscription-directsdk/tests/test_directsdk_probe_lifecycle.py
+++ b/claude-subscription-directsdk/tests/test_directsdk_probe_lifecycle.py
@@ -48,6 +48,27 @@ def test_a_timed_out_probe_leaves_no_child_and_no_scratch_dir(profile, tmp_path,
assert not os.path.exists(seen["cwd"]), "the scratch cwd was left behind"
+# A launcher that starts the real CLI and exits at once; the child keeps the probe's pipes open.
+LAUNCHER_EXITS = textwrap.dedent("""
+ import json, os, subprocess, sys
+ child = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(60)"])
+ open(os.environ["PROBE_LOG"], "w").write(json.dumps({"pid": child.pid}))
+ os._exit(0)
+""")
+
+
+@pytest.mark.skipif(os.name == "nt", reason="POSIX process group of an exited leader")
+def test_a_probe_whose_launcher_exited_leaves_no_child(profile, tmp_path):
+ import directsdk_setup
+ script = tmp_path / "claude.py"
+ script.write_text(LAUNCHER_EXITS, encoding="utf-8")
+ log = tmp_path / "probe.json"
+ env = {**os.environ, "PATH": os.defpath, "HOME": str(tmp_path), "PROBE_LOG": str(log)}
+ limits, _ = directsdk_setup.fetch_usage(command=[sys.executable, str(script)], env=env, timeout=2)
+ assert limits is None
+ assert not _alive(json.loads(log.read_text())["pid"]), "the launcher's child outlived the probe"
+
+
def test_an_exiting_interpreter_kills_a_probe_still_running_on_a_daemon_thread(tmp_path):
"""The `hermes status` shape: a daemon thread is inside discover_models when the main thread exits."""
plugin = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))Patch B: a non-text upstream setting refuses (#3)
diff --git a/claude-subscription-directsdk/directsdk_setup.py b/claude-subscription-directsdk/directsdk_setup.py
index 5e8667e..64607f0 100644
--- a/claude-subscription-directsdk/directsdk_setup.py
+++ b/claude-subscription-directsdk/directsdk_setup.py
@@ -573,7 +573,9 @@ def _loopback(host):
def _setting_text(key):
value = _setting(key, "")
- return value.strip() if isinstance(value, str) else ""
+ if value is not None and not isinstance(value, str):
+ raise ClaudeUpstreamConfigError(f"{key} must be text, not {type(value).__name__}; refusing to guess the route")
+ return (value or "").strip()
def _upstream_url(raw):
diff --git a/claude-subscription-directsdk/tests/test_directsdk_gateway.py b/claude-subscription-directsdk/tests/test_directsdk_gateway.py
index fc683bc..f75c7ed 100644
--- a/claude-subscription-directsdk/tests/test_directsdk_gateway.py
+++ b/claude-subscription-directsdk/tests/test_directsdk_gateway.py
@@ -150,6 +150,17 @@ def test_a_bad_gateway_setting_refuses_and_never_goes_direct(clean, tmp_path, mo
assert spawned == []
+@pytest.mark.parametrize("key", ["upstream_url", "upstream_proxy", "upstream_ca_file"])
+def test_a_non_text_gateway_setting_refuses(clean, key):
+ """`hermes config set ... upstream_proxy 3128` stores an int; it must refuse, not read as unset."""
+ import directsdk_setup
+ from hermes_cli.config import get_config_path
+ get_config_path().write_text(f"plugins:\n enabled: []\n entries:\n {PLUGIN_NAME}:\n settings:\n {key}: 3128\n",
+ encoding="utf-8")
+ with pytest.raises(directsdk_setup.ClaudeUpstreamConfigError, match=key):
+ directsdk_setup.resolve_upstream()
+
+
def test_an_inherited_base_url_stays_refused_with_the_opt_in(clean, tmp_path, monkeypatch):
_settings(upstream_url="https://gateway.example.com")
monkeypatch.setenv("ANTHROPIC_BASE_URL", "https://elsewhere.example.com")Patch C: routed-without-scope drops the inherited config dir (#2)
diff --git a/claude-subscription-directsdk/directsdk_setup.py b/claude-subscription-directsdk/directsdk_setup.py
index 5e8667e..2b216a8 100644
--- a/claude-subscription-directsdk/directsdk_setup.py
+++ b/claude-subscription-directsdk/directsdk_setup.py
@@ -206,7 +206,7 @@ def apply_profile_credential(env):
env.pop(OAUTH_TOKEN_ENV, None)
if routed and not selected:
# apply_config_dir read the launch environment; only this profile's own .env may choose.
- own = next(filter(None, (_scoped(get_secret, name) for name in _CONFIG_DIR_ENV)), None)
+ own = next(filter(None, (_scoped(get_secret, name) for name in _CONFIG_DIR_ENV)), None) if get_secret else None
if own:
env["CLAUDE_CONFIG_DIR"] = os.path.expanduser(own)
else:
diff --git a/claude-subscription-directsdk/tests/test_directsdk_profile_credential.py b/claude-subscription-directsdk/tests/test_directsdk_profile_credential.py
index 78a39f0..57dec8c 100644
--- a/claude-subscription-directsdk/tests/test_directsdk_profile_credential.py
+++ b/claude-subscription-directsdk/tests/test_directsdk_profile_credential.py
@@ -88,6 +88,20 @@ def test_a_served_profile_uses_its_own_gateway_headers_and_login_dir(ambient):
assert env == {"ANTHROPIC_CUSTOM_HEADERS": "X-Gateway: served", "CLAUDE_CONFIG_DIR": "/served/.claude"}
+def test_a_routed_task_with_no_scope_drops_the_inherited_login(ambient, tmp_path):
+ """A HERMES_HOME override alone (no scope bound, not multiplexed) has no reader: nothing inherited survives."""
+ from hermes_constants import reset_hermes_home_override, set_hermes_home_override
+ served = tmp_path / "served"
+ served.mkdir()
+ home = set_hermes_home_override(str(served))
+ try:
+ env = _apply({TOKEN: AMBIENT, "ANTHROPIC_CUSTOM_HEADERS": "X-Gateway: launch", "CLAUDE_CONFIG_DIR": "/launch/.claude",
+ "UNRELATED": "kept"})
+ finally:
+ reset_hermes_home_override(home)
+ assert env == {"UNRELATED": "kept"}
+
+
def test_a_single_profile_install_keeps_its_own_token(ambient, tmp_path):
"""Not multiplexed: get_secret reads os.environ, the token the child inherited anyway."""
_set_config_dir(str(tmp_path / "login"))| raise ClaudeUpstreamConfigError("upstream_url must be an https:// URL with no user:password@, ?query or " | ||
| "#fragment; refusing to send the Claude login anywhere else") |
There was a problem hiding this comment.
Nit: upstream_url accepts http:// for a loopback host, but this message says https only, and so do README.md:135, README.md:139 and plugin.yaml:59.
The gateway tests depend on the loopback exception for their plain-HTTP fixture, so keeping it is fine. The refusal text and the docs should say so, or someone debugging a local gateway is told something the code doesn't enforce. This changes only the wording. With it, tests/test_directsdk_gateway.py and tests/test_directsdk_profile_credential.py pass (35 passed). The README and plugin.yaml sentences need the same clause ("http:// only for a loopback host"):
| raise ClaudeUpstreamConfigError("upstream_url must be an https:// URL with no user:password@, ?query or " | |
| "#fragment; refusing to send the Claude login anywhere else") | |
| raise ClaudeUpstreamConfigError("upstream_url must be an https:// URL (http:// only for a loopback host) with no " | |
| "user:password@, ?query or #fragment; refusing to send the Claude login anywhere else") |
There was a problem hiding this comment.
Fixed in 6e2fefc. The refusal now says "https:// URL (http:// only for a loopback host)", and README and plugin.yaml carry the same clause.
…eter start The test ran the client with timeout=0.3 and read pids.jsonl once. The launcher fixture exits right after Popen of its descendant, and the descendant (a second `python -I`) needs about 0.3 s to write its pid record on an idle 8-core host and 0.3 to 0.8 s under load, so the record was often missing (`assert 1 == 2`). Not a branch regression: upstream main fails the same way under load (5/12 pass with 16 busy-loop processes; this branch 1/12 in the same run, with no difference in call-to-spawn time). The 0.3 s budget starts after spawn, so the pre-spawn work (profile credential, refresh-lock reap, extra-usage check) does not eat into it. The fixture launcher now exits only once its descendant has recorded itself, and the client timeout is 2 s so two interpreter starts fit. The descendant still sleeps 20 s, so a cleanup that waits for it still fails the bound (checked by turning kill_process_tree into a no-op: red, 21 s).
…process-tree helper) Greptile thread 4511161816. The probes had their own _kill_tree, which called poll() first: that reaps a wrapper that already exited (an npm shim, a shell launcher) and then skips the group kill, so the real CLI it started kept running after cleanup. communicate() reaped the leader the same way. own_process_group, unreaped and kill_process_tree move from directsdk.py into process_tree.py; the turn path and the probes both import them. The probe no longer uses communicate(): it writes stdin, reads stdout to EOF, and a timer kills the group on timeout, so the leader stays unreaped until kill_process_tree has signalled its group. Test: a /bin/sh launcher that backgrounds `sleep 30` and exits; after fetch_usage returns, the sleep is dead. Red on 980bdbb, green here.
… run on the launch login Greptile threads 4511161835 (security) and 4511161839. A routed profile with no CLAUDE_CODE_OAUTH_TOKEN and no config directory of its own lost the inherited CLAUDE_CONFIG_DIR, but the child still inherited the launch process's HOME, so Claude Code read ~/.claude there: the launch profile's login and allowance, the exact cross-profile use #313 is about. It now raises ClaudeCodeLoggedOut naming the two ways to give the profile a login (claude_config_dir, or CLAUDE_CODE_OAUTH_TOKEN in its .env). No profile-owned directory is guessed. ClaudeCodeLoggedOut moves to directsdk_setup so the setup probes can catch it; directsdk re-imports it. A routed task with no bound scope outside multiplexing (a HERMES_HOME override alone) got (None, True) from _secret_scope and then called _scoped(None, ...), a TypeError that the setup and /usage callers do not catch. It now raises core's UnscopedSecretError before any secret is read. Tests: routed + no token + no dir refuses and never spawns; routed + own dir uses it; HERMES_HOME override with no scope raises UnscopedSecretError. The three new tests are red on a369429.
… about the turn's own login Greptile threads 4511161845 and 4511161827. admit() treated a probe that could not tell (no get_usage answer, or an exported CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC that makes the CLI skip the fetch) as "not using extra usage" and sent the turn, so stop/ask could send a paid request without approval. Under stop/ask an unknown answer now refuses with the lookup's reason; nothing is cached, so the next request asks again. notify/allow never probe. An ask window the user already allowed skips the probe. The probe rebuilt the login through _child_env, so an explicit Client(env=...) was checked against the profile's login rather than the one the turn uses, and the answer was written under another account key, where admit then looked for it. check_extra_usage now runs the probe on a copy of the turn's prepared child env and the turn's upstream, and the probe returns its answer to admit, which records it under the turn's key (observe_usage returns what it recorded). The turn resolves its upstream once and the relay uses the same one. Tests: stop and ask with an empty probe answer refuse with the reason, never spawn, and probe again next time; an explicit Client(env=...) is probed with its own token and config dir. Both red on ab1625a.
…direct Greptile threads 4511161853 (security) and 4511352403. _setting_text turned any non-string value into "", so `upstream_proxy: false` or `upstream_url: 123` in config.yaml read as unset and resolve_upstream chose api.anthropic.com directly: an organization's required proxy or gateway was silently bypassed. A present non-string value now raises ClaudeUpstreamConfigError naming the setting. Absent or "" (or blank) stays the direct route. The upstream_url refusal, README and plugin.yaml said https only, but a loopback http:// URL is accepted (the gateway tests rely on it). They now say "http:// only for a loopback host". Tests: upstream_proxy: false, upstream_url: 123 and a list upstream_ca_file refuse in resolve_upstream and on a turn (red on 5360ac9); empty settings stay direct.
Greptile thread 4511161857. The legacy get_usage windows (five_hour, seven_day) send utilization as a 0..1 fraction, and usage_windows (/usage) already reads 1 as 100%. observe_usage compared the raw value to 100, so a full legacy window with extra usage on was recorded as not spilling and a cold stop/ask check admitted a paid turn. One helper, extra_usage.usage_percent, now normalizes both: /usage imports it (it was __init__._percent) and observe_usage uses it with fraction=True for the legacy windows. Test: legacy utilization 1 / 1.0 / 100 with extra usage on refuses under stop; 0.99 admits. 1 and 1.0 are red on 6554a55.
…he plan limits Greptile thread 4511161876. _usage_details iterated seven_day_breakdown.rows and model_scoped with `x or ()`, so a truthy scalar there (7, True, "abc") raised and /usage lost the whole snapshot, plan limits included. Both containers, and limits in the extra-usage policy's reading of the same body, are now used only when they are lists, as usage_windows already does for limits. Test: rows 7, model_scoped True, and both malformed keep the session and week windows; observe_usage does not raise on limits: 5. Red on 890e960.
…xy and CA Greptile thread 4511161868 (verify). Measured with the real CLI, 2.1.319 and the 2.1.292 CI pin, behind an opted-in loopback gateway and a recording CONNECT proxy (out/setup-round1/probe_paths.py): get_usage and the initialize handshake send nothing to ANTHROPIC_BASE_URL. Their fetches (/api/oauth/usage, /api/claude_cli/bootstrap, /v1/mcp_servers, /api/eval/sdk-*, /api/event_logging/v2/batch, plus account/settings and grove on 2.1.292) go to api.anthropic.com directly, with the login's Authorization. A turn sends only POST /v1/messages to the base URL, and the relay forwards only that path. So the bearer never goes to any host but the gateway (Messages) and Anthropic (the CLI's own calls). The paths are already pinned by the relay. The gap was egress: with upstream_proxy / upstream_ca_file set those direct calls ignored them, so an org that only allows egress through its proxy lost /usage, and its TLS-inspecting CA was not trusted. The child now gets HTTPS_PROXY (with the CLAUDE_SUBSCRIPTION_DIRECTSDK_PROXY_AUTH credentials) and NODE_EXTRA_CA_CERTS from those settings, with the loopback relay added to NO_PROXY. Verified on the real CLI with only the plugin settings set: every direct call then went through the recording proxy and /usage still answered. README and plugin.yaml now say which calls go where. Tests: the probes and a turn start the CLI with the proxy, CA and NO_PROXY (red on e5d3e5d); the relay answers 404 to the four non-Messages paths without touching the gateway and forwards /v1/messages (guards existing behaviour).
|
@greptileai please re-review. Round 1 fixes are in 980bdbb..e48ec17, one commit per finding group; each thread has its sha. One extra commit (f6b335a) fixes a flaky lifecycle test. |
Greptile thread 4512444851. _probe read stdout until EOF and relied on the timeout's kill to close it. The kill cannot reach every holder: on Windows kill_process_tree skips a launcher that already exited, and its Claude Code child keeps the inherited stdout open, so /usage or the picker blocked past the timeout (past core's 10 s /usage hook deadline). The read now runs on a daemon thread joined with the timeout; on expiry the probe raises TimeoutExpired and the finally still kills and waits for the tree. Tests: a launcher whose detached descendant holds stdout returns "did not answer in time" inside the timeout (red on e48ec17: blocks for the descendant's 20 s); /usage against a CLI whose stdout never closes answers under 10 s. The second replaces an assertion that spied on threading.Timer.
… absolute, one spelling on Windows Greptile threads 4512444862 and 4512444873, and the Windows CI lane on e48ec17. - upstream_ca_file was loaded by the relay relative to Hermes' cwd but passed to Claude Code relative in NODE_EXTRA_CA_CERTS, and the CLI runs in a scratch cwd, so a relative path trusted nothing there. It is resolved to an absolute path once and both use that. - urlsplit's hostname drops an IPv6 literal's brackets, so http://[::1]:3128 reached the CLI as http://::1:3128. The brackets are restored. - Windows env names ignore case, so setting HTTPS_PROXY and https_proxy (and NO_PROXY / no_proxy) wrote one variable twice; the child got one of them and the test's lookup of the lowercase key failed. Windows now sets one spelling, POSIX keeps both. Tests: the proxy-and-CA test uses a relative CA path from a chdir'd cwd and counts spellings per platform (red on e48ec17: the relative path reaches the child); an IPv6 proxy reaches the CLI as http://[::1]:3128 (red on e48ec17).
|
@greptileai please re-review. Round 2 fixes are e3a3d60 (probe read deadline) and 8966b57 (absolute CA path, IPv6 proxy brackets, one proxy spelling on Windows, which is the Windows CI failure). Each thread has its sha. |
…o kill reaches subprocess.run Windows CI on 8966b57. The fake reported poll() None, so on Windows the probe's cleanup ran kill_process_tree's taskkill through subprocess.run, which the test had replaced with this fake Popen; that raised AttributeError, and /usage answered "gave no usage answer" instead of the timeout. The fake now reports an exited leader whose stdout stays open, which is the Windows case e3a3d60 fixes (the kill skips an exited launcher). Still red on e48ec17 (blocks past the deadline), green here.
|
@greptileai review (f12af83 is a test-only fix for the Windows CI lane) |
The PR added 42 test functions of its own beside the 16 salvaged with contributor commits. Most pinned one rule many times over (a parametrize row per setting, a test per refusal message) or drove the same path as a sibling. Merged those into one test per user-visible behaviour: /usage detail, the extra_usage modes, the fresh-process usage check, the org gateway route, its proxy/CA egress, a bad setting refusing, the CA trust, probe cleanup, the refresh-lock reap, and a served profile's own login. Contributor tests are unchanged. Each fix commit's code, reverted alone on this tree, still fails at least one remaining test.
|
@greptileai review |
# Conflicts: # claude-subscription-directsdk/plugin.yaml
|
Thanks for salvaging #314 and closing the two leaks it left (the inherited token for a served profile with no login of its own, and the swallowed |
unsupportedpastels
left a comment
There was a problem hiding this comment.
Review: PR #357 fix(directsdk): telemetry off, full /usage, own login per profile @ a8b0fa89
Verdict: 🟡 Changes requested. I found no production regression in the code: the changed paths passed the focused suite and offline native qualification, and the four findings are small follow-ups. I'm holding the approval only on verification: the tests workflow has not run at this head on any OS (the Windows and macOS paths here are unexercised) and the branch conflicts with main. A rebase and one green matrix run would settle it.
Direction: The telemetry and login changes address #331 and #313, and the explicit gateway opt-in implements #46 without accepting an inherited ANTHROPIC_BASE_URL. The paid-overage policy is useful but only a subset of #43; I'd keep that issue open. Forwarding the bearer to an explicitly selected gateway remains a maintainer policy decision under AGENTS.md, not something this code review decides.
| Critical | Major | Minor | CI |
|---|---|---|---|
| 0 | 0 | 4 | — none |
Checked:
- Compared the PR's own delta against
85e835c3: provider registration/account rendering, request preparation and receive loop, setup/discovery/usage probes, extra-usage policy, relay changes, process-tree extraction, configuration and changed tests. Tracedcreate_client → Client.create → login preparation → usage admission → relay → native, plus the setup and account-usage hooks. - Linux/Python 3.14: the nine focused test files passed: 118 passed, RUNNER_RC=0. The initial complete runs produced 5 failed, 557 passed at this head and 5 failed, 509 passed at the merge base, each with nine passing subtests. The failing IDs match; attribution is below.
- With
USERrestored and the four inherited refusal-fixture cases deselected, the head run passed: 560 passed, 4 deselected, 9 subtests passed, RUNNER_RC=0. This includes two additional positive probes, not 560 unchanged plugin tests. There is notestsworkflow run at this SHA; the GitHub checks are Greptile checks only. - Default configuration is telemetry off,
extra_usage: notify, and no gateway. Existing single-profile and explicit-client login paths, tool/text replies, model discovery and usage windows remain exercised. A two-thread/two-client probe delivered both replies using distinct scoped fixture tokens, without changing the launch environment; that same identity assertion fails at the merge base. A separate probe confirmed that the spend check retains an explicit client's prepared login and route. - Both offline evals passed with real Claude Code 2.1.292 and 2.1.321, using synthetic loopback responses and no account login. They exercise tool/text/refusal/error/cancellation admission and cache-wire replay; the cache eval recorded 22 requests per version. These are protocol checks, not paid-model or subscription-meter measurements.
- Mutation checks caught changes to telemetry defaults, usage detail retention, token precedence, extra-usage admission, gateway selection, probe cleanup, stale-lock removal, metered-login rejection and relative-HOME resolution. The missing fresh-lock negative control is m4.
- The trial merge with main
8b5b6296conflicts only inplugin.yaml's version: 0.4.0 versus 0.3.5. This is a mechanical version conflict, not a PR defect. Contributor authorship and the lock-reap coauthor trailers are preserved.
Not checked: Windows/macOS execution, real subscription logins, the claimed 0.25-second spawn improvement, live picker cardinality, actual paid-overage events, Team/Enterprise billing responses, and the attached screenshots. Windows exited-launcher cleanup remains source-only; a detached stdout holder is explicitly outside the process-group cleanup guarantee. Abrupt owner SIGKILL leaves a usage child alive both before and after this change; no new guarantee is claimed here.
Security: Checked argv/subprocess inputs, scoped credentials and child environments, login-directory and refresh-lock handling, loopback relay boundaries, gateway opt-in and TLS verification against the plugin's documented promises and core SECURITY.md §§2.2, 2.3, 2.5, 2.6 and 3.1–3.2. No new credential-boundary failure was demonstrated in the tested paths. See m4 for a safety-regression test gap.
Thanks @alt-glitch! Keeping login preparation shared across turns and probes is worth preserving.
Critical — none
Major — none
Minor
- m1
directsdk_setup.py:50–52: The new metered-login hint loses its switching instructions in interactive setup, confirming Greptile's4517763274. Core's TTY login gate prints onlydetail.split('.')[0]; with a nativeapi_keylogin and a failed login command, the observed output explains API billing, then printsStarting auth loginandLogin failed, but never says to use the Claude account. The non-TTY control includes the instruction. I'd join the first two sentences with a semicolon so both survive that renderer. This retained rejection of API-key/cloud logins and acceptance of subscription/setup-token logins: the unchanged setup and credential suites passed 42 tests on the suggested wording, and the rendering probe showed the instruction in both modes.
Verified wording change
--- a/claude-subscription-directsdk/directsdk_setup.py
+++ b/claude-subscription-directsdk/directsdk_setup.py
@@
-METERED_HINT = ("Claude Code would sign in with {0}, which bills API usage instead of your Claude subscription. Run `claude auth login` "
+METERED_HINT = ("Claude Code would sign in with {0}, which bills API usage instead of your Claude subscription; run `claude auth login` "-
m2
AGENTS.md:27–30and97–99: Still open from BearHuddleston's earlier review, confirmed at this head: the picker rule omits the new gateway exception, and the contributor rule does not record the explicitly opted-in bearer-forwarding policy.discover_modelsnow selects the relay wheneverupstream.opted_in, including proxy-only or CA-only configurations. I'd document that precise exception and have the maintainer record the policy decision rather than leave the implementation and contributor instructions disagreeing. -
m3 PR description, opening paragraph: I'd change
Closes #43toRefs #43. That issue requests configurable reserves for independent/model-specific windows, cross-process coordination and lifecycle receipts. This implementation intentionally supplies process-local paid-overage handling instead;extra_usage.py:101–119checks account-wide full windows and138–180admits/refuses from in-memory state. Those are narrower semantics, not completion of the requested reserve-policy contract. -
m4
tests/test_directsdk_probe_lifecycle.py:117–134: Add a fresh-lock negative control. Removing the 60-second age predicate fromreap_orphaned_refresh_lockstill leaves all 6 lifecycle tests passing. The current code correctly keeps a fresh empty lock; the gap is that a future edit could remove an active refresh lock unnoticed. I ran the following exact extension: 2 passed at this head; 1 failed, 1 passed at the merge base because its stale lock remains; 1 failed, 1 passed with the age-guard mutation because its fresh lock disappears. Ordinary replies still complete in both cases.
Test extension exercised at the merge base, head and age-guard mutation
--- a/claude-subscription-directsdk/tests/test_directsdk_probe_lifecycle.py
+++ b/claude-subscription-directsdk/tests/test_directsdk_probe_lifecycle.py
@@
-def test_a_turn_reaps_the_orphaned_lock_of_its_own_login(profile, tmp_path, monkeypatch):
+@pytest.mark.parametrize("age", [0, 120], ids=["fresh", "stale"])
+def test_a_turn_reaps_the_orphaned_lock_of_its_own_login(profile, tmp_path, monkeypatch, age):
@@
- lock = _lock(config, 120)
+ lock = _lock(config, age)
@@
- assert not lock.exists()
+ assert lock.exists() == (age < 60)The four refusal-after-tool failures are inherited core/test-stub drift (reason keyword); main's merged #370 fixes that stub. The fifth initial failure is the harness removing USER while the workspace test deletes POSIX os.getuid: restoring USER makes that exact test pass at both revisions. Neither is attributed to this PR; rebasing also picks up #370.
Good
- A routed profile without its own login is refused before spawn, while independently scoped tokens can produce simultaneous successful replies.
- Unknown usage answers in
ask/stoprefuse without enabling paid fallback; defaultnotifyretains working turns. - Probes share process-group ownership with turns, and the stdout deadline no longer depends on descendant termination succeeding.
Next steps
|
@greptileai review |
…ed setup shows its instruction Greptile on #357: under an opted-in upstream_proxy an inherited NO_PROXY (* or api.anthropic.com) let Claude Code's own account calls skip the organisation's proxy; the child now gets only the loopback relay as a bypass. And a login that bills API usage returned a login command, so Hermes setup ran claude auth login and printed only 'Login failed'; it now returns no command and setup prints the whole instruction.
|
@greptileai review |
The Claude subscription plugin (DirectSDK) now keeps Claude Code's telemetry off by default, shows the whole Claude account on
/usage, runs each profile on its own Claude login, and adds per-profile settings for paid extra usage and an organization gateway. Closes #331, #313, #43, #46, and the plugin side of NousResearch/hermes-agent#146288.Before
Plugin 0.3.5, the version on
main:claude_code_telemetrysetting off, and turning it off cost the picker most of its models and/usageits limits./usageshowed the plan windows and dropped the rest of the account answer.flowchart LR T[turn on a served profile] --> L{own Claude login?} L -->|no| G[the launch profile's login and allowance] R[Claude Code rate_limit_event] --> X[dropped]After
Plugin 0.4.0:
/usagekeeps its limits.claude_config_dir. With neither, the turn is refused before Claude Code starts, and the message names both ways to give the profile a login.extra_usage(allow,notifydefault,ask,stop) acts on the rate-limit event and, in a fresh process, on one zero-inference usage check. State is in memory only.upstream_url,upstream_proxy,upstream_ca_fileare one explicit opt-in. Only Messages requests go to the gateway; Claude Code's own account calls take the same proxy and CA. A bad value refuses and never falls back to the direct route.flowchart LR T[turn on a served profile] --> L{own token or claude_config_dir?} L -->|yes| O[runs on that login] L -->|no| F[refused before Claude Code starts] R[Claude Code rate_limit_event] --> P{extra_usage} P -->|notify| W[one warning per window] P -->|ask or stop| S[next request refused until allowed or reset]What the user sees
hermes usagenotify: one warning per window.ask/stop: the next request is refused before it is billed, naming the reset time and thehermes config setline/usageand the pickerhermes statuson Windowsclaude.exeleft running.oauth_refresh.lockleft by a crashed Claude CodeThe same account and Claude Code; the after rows are the detail
get_usagealready returned.A profile with no Claude token or config directory of its own, served by the shared gateway. Before, it answered on the launch profile's login.
Not in this PR
notify: the warning goes to the log. Core has no provider-to-UI notice seam yet.hermes-agent(unchanged on purpose).hermes statusfix: covered on Linux with a real child on a daemon thread at interpreter exit.Live verification
main)hermes -p <scratch> usage, real Claude Code 2.1.319, zero inferenceExtra usage: off,Usage credits spent: 0.00 USD,This week by surfacehermes -p <served profile> chat -q, a profile with no login of its own, served by the shared gatewayokon the launch profile's logindirectsdk_admission,directsdk_cache_wireon Claude Code 2.1.319 and the CI pin 2.1.292Gateway wire check, on real Claude Code 2.1.319 and 2.1.292 behind a loopback gateway and a recording CONNECT proxy:
get_usageand the picker'sinitializerequest send nothing to the base URL.POST /v1/messagesto the gateway.Not verified live: a real
isUsingOverage: trueevent (no account here is past its limit) and the Team/Enterprise 400.Salvaged by cherry-pick with authorship kept: #258 (@ephyria-k), #314 (@kalfian), #281 (@salch-cred), #27 (@grantjayy), #28 (@100yenadmin); #25 the lock reap only, with Co-authored-by. Superseded: #337, #282, #257, #130, #23.
Follow-ups
notifycan show in the chat.Next PR
#360 (tool calls stream as written) is stacked on this branch and moves to
mainwhen this merges.