Repository navigation
fix(providers): surface OpenCode Zen free-tier refusal as actionable error - #716
Conversation
- run_agent_cli maps the Zen FreeTierError envelope (403 under the deny-all isolation, verified by env bisect, re-observed live 2026-10-03) to a fail-closed AgentCLIError naming the incompatibility and suggesting a key-backed model; plain non-zero exits keep the generic message - No sandbox weakening: refusal never falls back to a weaker policy - Verified: 2 new cross-platform tests (fake Popen); file suite 43 passed / 3 skipped; ruff check + format + diff-check clean Signed-off-by: Yoseph Zuskin <zuskinyoseph@gmail.com> Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <noreply@opencode.ai>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Hi @Yoseph-Zuskin, thank you for bisecting the Zen free-tier refusal and replacing a confusing exit-code error with useful guidance, without loosening the sandbox!
Value and readiness: This delivers what #715 asks for. Suppose opencode exits non-zero and the first 2,000 bytes of stdout contain FreeTierError or "can only be used from within OpenCode". run_agent_cli then raises an AgentCLIError that names the conflict with the sandbox and suggests a key-backed model. Every other non-zero exit keeps the generic message. Both paths still fail closed, and nothing falls back to a weaker policy. The check is limited to binary_name == "opencode". The tests cover both acceptance criteria and CI is green. Ready for final maintainer review.
Material findings
- [Non-blocking]
src/skillspector/providers/_agent_cli.py:1253-1263: the markers are matched as plain substrings anywhere in the first 2,000 bytes of stdout. With--format json, stdout also carries the model's streamedtextevents, and the prompt contains untrusted skill content. A skill could make the model echo "FreeTierError". If the run then fails for an unrelated reason, the user is wrongly told to switch models. The call still fails closed, so only the diagnosis is affected. Matching only{"type":"error"}events, the envelope your test uses, would close this. An optionalCliSpechook for failure classification would also keeprun_agent_cliCLI-agnostic, as the module docstring and the "HOW TO ADD A NEW AGENT CLI" note (:934) promise. - [Non-blocking]
src/skillspector/providers/_agent_cli.py:464-468: the comment states as permanent fact that Zen answers 403 to any call with the deny-all isolation. The record is mixed. #575 ranopencode/mimo-v2.5-freeunder this isolation on 2026-09-18. #613 hit the 403 on 2026-09-22. This PR reports mimo working under full isolation on 2026-10-03, before the refusal came back. Please date the observation, so readers treat it as Zen's current policy rather than a guarantee. - [Non-blocking] The new message says "Use a key-backed model instead" but not how to choose one. Naming the setting, for example "set
SKILLSPECTOR_MODELto aprovider/modelyou have credentials for", would make it fully actionable.
PIC tradeoffs: None identified. The PR deliberately refuses to relax the isolation for free-tier models, which matches the provider's fail-closed contract.
Verification and gaps:
- Traced
run_agent_cliat this head. The overflow and timeout checks still run first. The new branch only changes the message of anAgentCLIErrorthat was already raised.stdout_snippetis used only for matching and never appears in the message. - Tests (
tests/provider/test_opencode_cli.py:588-614):_FakePopendrives the real_run_bounded, andpreflight=Noneis set throughdataclasses.replaceon the frozen spec. The refusal envelope produces the new message, andb"boom"keeps "exited with code 1". Two cases are not covered (optional): a marker beyond 2,000 bytes, and a non-opencode binary that prints the marker. - I could not confirm Zen's server behaviour. Per policy, I did not run the tests or OpenCode.
- CI: all six checks pass. The PR merges cleanly with current
main, #737, #738, #747 and #748. - Overlap: it conflicts with #713 in the constants block. #713 replaces
_OPENCODE_SUPPORTED_VERSION, and this PR inserts_ZEN_FREE_TIER_MARKERSright after that line. The fix is mechanical: keep #713's bounds, then add the markers. I suggest landing this PR first, since #713 needs other changes and can rebase onto it.
Decision: Approved (reviewed head 56b900328f61568426acc130f15ef17e9f57dbb1)
Fixes: #715
Problem
Zen answers every
opencode_cliinference call withFreeTierError403 ("free tier can only be used from within OpenCode")whenever the provider's deny-all isolation is present — so all semantic
scans via Zen free models fail 0/N with the opaque message
opencode exited with code 1; stderr=''and no hint of the cause.Bisected to the single variable (each a live Zen call, sequential):
OPENCODE_CONFIG_CONTENTalone → 403;OPENCODE_PERMISSIONalone → 403OPENCODE_CLIENT, file-based policy (opencode.jsonin theredirected config dir), paths/disables env groups → still 403
opencode/nemotron-3-ultra-freeandopencode/mimo-v2.5-freeboth 403 under isolation, both OK without(mimo worked under full isolation on 2026-10-03, so this is a Zen-side
policy change, not a provider regression)
Conclusion: ANY permission-deny trips the gate — the deny-all sandbox
and Zen free tier are mutually exclusive until Zen's policy changes.
Approach
Fail closed with guidance instead of the opaque exit-code message.
run_agent_clinow matches theFreeTierErrorenvelope on a non-zeroexit and raises an actionable error (free-tier models incompatible
with the required sandbox; use a key-backed model or retry if Zen
policy changes). Deliberately NO silent fallback to a weaker sandbox.
Verification
envelope → actionable error; plain non-zero exit → unchanged generic
message
tests/provider/test_opencode_cli.py: 43 passed / 3 skippedruff check+ruff format --checkclean,git diff --checkcleanRisks
are a tuple (
FreeTierError,can only be used from within OpenCode)so either signal fires, and a miss degrades to the previous generic
message (never to a wrong success).
Signed-off-by(maintainer: verify on push).