feat(gauntlet): add Gauntlet Loop plugin

[se-jo-ma]

Implements Matt Shumer's Gauntlet Loop as a plugin.

A lead agent splits a goal into the smallest independently judgeable pieces. Each piece gets a builder and a separate critic. The critic compares the work blind against a concrete external bar and names the biggest remaining gap; that gap goes back to the builder. Repeat until you stop it.

Enforced, not just asked for

The two load-bearing rules fail silently when they are only prompt text, so they are wired into scripts:

Rule	Mechanism
No loop without a bar	ab.py stage refuses unless .gauntlet/bar.md records PASS on all five soundness checks (inspectable, external, held-out, discriminating, un-gameable)
The critic is blind	Candidate and reference shuffled into A//B/; mapping written outside the trial dir at mode 0600
The critic stays blind	PreToolUse hook denies any tool call touching .gauntlet/keys/; ab.py reveal reads the key in-process so the lead can still resolve
Artifacts, not arguments	Staged trials contain only A/, B/, and a generated brief — no builder notes or diffs
No re-grading	ab.py verdict refuses to overwrite an existing verdict
Receipts	Every staged file sha256'd into an append-only ledger.jsonl

Nothing constrains what the lead builds or how it splits the work. The determinism is in the gate, never in the generation.

Two leaks found and closed during the build

1. Filename leak — A/ours.png next to B/cod_reference.png identifies our side from the path alone. stage now refuses on path-set mismatch and cleans up the half-staged trial (--allow-name-mismatch to override).
2. Board server leaked the mapping — SimpleHTTPRequestHandler rooted at .gauntlet/ served keys/*.json over HTTP on all interfaces, defeating the hook. Now 403s keys/ (incl. URL-encoded and traversal) and binds localhost; --host 0.0.0.0 is explicit opt-in.

Domain bar recipes

Recipes for finding a bar, not menus of preset bars:

• bars-agentic.md — judge trajectory + end state. Verifier leakage, task memorisation, pass@1 vs best-of-n, harness cheating, cost blindness.
• bars-rl.md — judge frozen-protocol eval rollouts. Protocol/env drift, seed cherry-picking (≥5 seeds, median+IQR or IQM), tuning on eval, shaped-reward inflation.
• bars-coding.md — differential testing against a reference impl. Builder-written tests, test mutation, overfit fixtures, mock leakage.

Surface

Commands /gauntlet:new, :bar, :run, :status. Agents lead, builder, critic, bar-scout, smoother.

Verified

make lint green. Scripts smoke-tested end to end: bar gate refuses missing/unsound bars and names the missing checks; A/B assignment non-degenerate over 12 stages (5 A / 7 B); key-guard blocks Read/Bash/Grep and allows reveal; re-grade refused; filename-mismatch refused with cleanup; board serves board.html 200 and keys/ 403 on every bypass tried.

Note

Also syncs .cursor-plugin/marketplace.json, which had drifted from the Claude one (missing ui-fidelity) and was failing scripts/validate-marketplaces.sh on main, and adds the missing ui-fidelity rows to the README.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XexkC5VGxU2G9rTNDTwhGJ