Skip to content

fix: stop /tokens from permanently ratcheting the REPL token cap down - #145

Open
shrdgn wants to merge 1 commit into
mainfrom
claude/cli-tokens-ratchet-fix
Open

fix: stop /tokens from permanently ratcheting the REPL token cap down#145
shrdgn wants to merge 1 commit into
mainfrom
claude/cli-tokens-ratchet-fix

Conversation

@shrdgn

@shrdgn shrdgn commented Aug 16, 2026

Copy link
Copy Markdown
Owner

What & why

openfusion chat's REPL /tokens N command (cli.py:364-369) applied apply_overrides()
against its own current (possibly already-narrowed) config instead of the original loaded
config. apply_overrides's max_tokens handling goes through _narrow(), which is
intentionally narrow-only (min(configured, requested)) — that's correct for the HTTP
per-request-override path, where a client must never widen a server-set ceiling. But reusing
it against the REPL's own running config meant that once a user typed /tokens 100, a later
/tokens 5000 stayed silently capped at 100 for the rest of the session — the cap could only
ever go down, never back up:

after /tokens 100:            100
after /tokens 5000 (from c1): 100   # still 100, not 5000 — the bug

This is a real, user-visible UX bug: answers get truncated indefinitely with no error message
explaining why, until the REPL is restarted. It went uncaught because run_chat — the entire
REPL loop including all /-commands — had no direct test; the existing CLI tests only exercise
_chat_turn and dispatch to run_chat, never the command loop itself.

Separately, switching /preset (cli.py:357-363) already rebuilt correctly from
base_config, but doing so silently discarded any active /tokens override — so the two
commands didn't compose correctly with each other either.

Fix: /tokens and /preset now both recompute the active config from the original
base_config via a small _rebuild(preset, tokens) closure, tracking the currently-active
preset and token override as REPL-local state. Each command re-applies both overrides on
top of the base config, so either command can be issued in either order, any number of times,
without the other's effect being lost or becoming a ratchet.

Also fixed a related, smaller correctness bug found in the same area: the startup summary's
tools: line (_summarize_config, cli.py:43) reported web search+fetch whenever
web_search was enabled, regardless of web_fetch's actual (independently configurable)
value — and never reported a web_fetch-only configuration (web_search: false, web_fetch: true)
at all, always showing off for it instead.

Found via an automated repo-review scheduled task; verified by hand by reading cli.py's
REPL command handlers and overrides.py::_narrow, and reproducing the ratchet with a scripted
REPL session before fixing it.

How it was tested

  • ruff check . passes
  • pytest -q passes (494 passed, no live network)
  • New behavior has a test — tests/test_cli.py::test_tokens_command_can_raise_the_limit_back_up
    and test_preset_switch_preserves_an_active_tokens_override drive the actual run_chat REPL
    loop (monkeypatching rich.console.Console.input and cli._chat_turn) and assert the
    effective panel_max_tokens seen by each simulated turn. Also added
    test_summarize_config_reports_fetch_only_when_search_is_off and
    test_summarize_config_reports_search_only_when_fetch_is_off for the banner fix.
  • Docs updated — not needed, no config/request-surface/defaults changed, only REPL/CLI
    session-local behavior
  • No secrets, prompts, or response bodies added to logs or metrics
  • Bench number — not applicable, no quality/cost/behavior change to model calls

Notes for reviewers

Only run_chat's REPL command handlers and _summarize_config's tools label are touched.
apply_overrides/_narrow themselves are unchanged — they're correct as-is for the HTTP
per-request-override path this bug isn't part of; the fix is entirely in how the CLI's REPL
loop composes its own local state before calling into them.


Generated by Claude Code

The chat REPL's /tokens N command applied apply_overrides() (a narrow-only
transform) against its own already-narrowed running config instead of the
original loaded config, so raising the cap back up (e.g. /tokens 100 then
/tokens 5000) silently stayed at the lower value for the rest of the
session. /tokens and /preset now both recompute from the base config and
compose with each other. Also fixes the startup banner's tools: line, which
ignored web_fetch whenever web_search was on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SfTJhjkqkDfTMzufuBcLtQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants