Skip to content

Persist each lane's answer as it lands, not after the last one (#58) - #59

Open
CryptoJones wants to merge 1 commit into
mainfrom
fix/persist-lane-answers-incrementally
Open

Persist each lane's answer as it lands, not after the last one (#58)#59
CryptoJones wants to merge 1 commit into
mainfrom
fix/persist-lane-answers-incrementally

Conversation

@CryptoJones

Copy link
Copy Markdown
Owner

Fixes #58 — the "roundtable randomly dies" bug.

The problem

main() collected the fan-out with results = list(pool.map(...)) and wrote the transcript only afterwards. Every answer was held in memory until the last lane returned, so any interruption during the fan-out discarded all completed lanes — including paid http lanes that had already billed for tokens nobody would ever read.

It presents as a run that dies with zero output and no transcript, and it bites hardest when it costs most: a long brief with many lanes, where the slowest lane decides whether the other eleven survive. Observed tonight on a 7-lane, 92 KB brief — stopped mid-flight, 0 bytes out, no transcript, after every lane had been dispatched.

The change

  • Claim the transcript path before the fan-out
  • Drive it with as_completed and rewrite the transcript after every lane
  • Write each result into the slot its lane was dispatched from, so ordering is preserved and a partial transcript reads exactly like a complete one, with null for lanes still out
  • Atomic writes (.tmp then replace), so a kill during the write leaves the previous good transcript instead of a truncated one
  • Partial transcripts carry "partial": true plus a note; the final write clears the marker and attaches the synthesis

Behaviour on a successful run is unchanged.

Verification

  • kill -9 mid-run now leaves a transcript holding the two lanes that had answered, third slot null, partial: true. Before the change that kill left nothing.
  • Completed run: no partial marker, synthesis and collisions attached, no stray .tmp.
  • Suite green — 119 passed, 29 subtests.
  • Verification-set items 1 and 2 re-run by hand: --list shows the correct route per lane with zero network calls; a full run has every lane answer with exit 0.

Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/

A run interrupted anywhere in the fan-out discarded every answer it had
already collected. `list(pool.map(...))` blocks until the last lane returns,
and the transcript was written only after that, so all answers lived in memory
for the whole run. A SIGKILL, a closed terminal, a harness timeout or a Ctrl-C
threw the lot away — including paid http lanes that had already billed for
tokens nobody would ever read. It presents as a run that "randomly dies" with
zero output and no transcript, and it is worst exactly when it costs most: a
long brief with many lanes, where the slowest lane decides whether the rest
survive.

The transcript path is now claimed before the fan-out and rewritten after every
lane completes, driven by as_completed. Each result is written into the slot its
lane was dispatched from, so ordering is preserved and a partial transcript
reads like a complete one with null for the lanes still out. Writes go to a
.tmp and are replaced into place, so a kill mid-write leaves the previous good
transcript rather than a truncated one. Partial transcripts carry "partial":
true and a note; the final write clears it and attaches the synthesis.

Verified: kill -9 mid-run now leaves a transcript holding the two lanes that
had answered, third slot null. A completed run is unchanged — no partial
marker, synthesis and collisions attached, no stray .tmp. Suite green
(119 passed, 29 subtests). Verification set items 1 and 2 re-run by hand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VmauK5UCYGRYyXoQ36FQ3S
@coderabbitai

coderabbitai Bot commented Aug 29, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2f3c3cbd-b102-40e2-9151-eb6d46fb10ed


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A run that dies mid-fan-out loses every answer it already collected

1 participant