Skip to content

Refuse to run the whole panel in one invocation; add --each - #60

Merged
CryptoJones merged 1 commit into
mainfrom
fix/one-run-per-lane
Aug 30, 2026
Merged

Refuse to run the whole panel in one invocation; add --each#60
CryptoJones merged 1 commit into
mainfrom
fix/one-run-per-lane

Conversation

@CryptoJones

Copy link
Copy Markdown
Owner

The bug this prevents

A single roundtable invocation finishes when its slowest lane finishes. That
couples every lane's fate to the worst one:

  • one slow lane sets the wall-clock for all twelve;
  • raising that lane's timeout raises the whole run's duration, pushing it past the
    caller's timeout;
  • a lane that dies cannot be retried without rerunning everything — including http
    lanes that already billed for tokens nobody will read.

Lanes are supposed to be independent. Sharing a deadline makes their failures
dependent, which is the opposite of the point of a multi-model panel.

Measured

The poolside/ACP lane needs ~400s alone on a 110KB brief. Under 12-lane contention it
exceeded 780s and was lost in four consecutive rounds. Each loss was avoidable: run
alone, with no contention and no shared deadline, the same lane answers fine. Raising
its ceiling only made every other lane wait longer.

What changes

--each     run each selected lane as its own process (the right default)
--lanes N  one lane, unchanged
--panel    the old shared-deadline behaviour, for anyone who means it

A bare multi-lane call is now refused, with those three options printed in the
error. Deliberate: the wrong call was the obvious-looking one, so the tool makes it
unavailable rather than merely documenting against it.

--each re-invokes the script once per lane — separate process, separate deadline,
separate transcript.

Tests

Three end-to-end tests exercise pacing, per-vendor semaphores and silent-lane
detection, which genuinely need the shared-deadline path — that is what --panel
names, and the test helper now passes it for multi-lane cases. 119 tests pass.

Docs updated: README.md usage block, AGENTS.md/CLAUDE.md contributor guide, and
skill/SKILL.md.

Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/

A single invocation finishes when its slowest lane finishes, which couples every
lane's fate to the worst one. One slow lane sets the wall-clock for all of them.
Raising that lane's timeout raises the whole run's duration, pushing it past the
caller's timeout. And a lane that dies cannot be retried without rerunning
everything, including http lanes that already billed for tokens nobody read.
Lanes are supposed to be independent; sharing a deadline makes their failures
dependent.

Measured 2026-08-30: the poolside/ACP lane needs ~400s alone on a 110KB brief but
exceeded 780s under 12-lane contention. Four consecutive rounds lost its answer.
Every one of those losses was avoidable -- the same lane run by itself, with no
contention and no shared deadline, answers fine.

  --each     run each selected lane as its own process (the right default)
  --lanes N  one lane, unchanged
  --panel    the old shared-deadline behaviour, for anyone who means it

A bare multi-lane call is now refused, with those three options in the error.
This is deliberate: the wrong call was the obvious-looking one, so the tool now
makes it unavailable rather than merely documenting against it.

The three end-to-end tests that exercise pacing, per-vendor semaphores and
silent-lane detection genuinely need the shared-deadline path, which is what
--panel names; the test helper now passes it for multi-lane cases. 119 tests
pass.

Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VmauK5UCYGRYyXoQ36FQ3S
@CryptoJones
CryptoJones merged commit 057b49f into main Aug 30, 2026
4 checks passed
@CryptoJones
CryptoJones deleted the fix/one-run-per-lane branch August 30, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant