feat(builder): forced self-refutation checklist before declaring done#252
Merged
Conversation
…#247) The builder repeatedly shipped tests that go green without binding the objective — a Jepsen leg with no nemesis (wyrd #257 Iter-1), a fixture excluding the killed node (#366 Iter-2), a test driving MemMeta instead of the real backend (#366 Iter-3) — each caught only by the adversary at the cost of an iteration. builder.md already forbids this in prose but forces no check. Add a "Before you declare done" step: Do reverts the fix to confirm a genuine red, confirms the test drives production not a stand-in, and confirms the fixture includes the failing element — recording all three answers in build-notes.md so the human sees them at sign-off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dbccef2c41
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…review)
Codex flagged that question (a) "Does the test still pass with your fix reverted?" has
the desired answer "no" for a valid regression test, which contradicts the rule "any no
means not done". Rephrase all three questions so a sound fix answers yes ("Does the test
fail with your fix reverted?"), making the not-done rule coherent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #247.
Problem
The builder repeatedly shipped tests that go green without binding the objective — each caught by the adversary reviewer at the cost of an iteration: a Jepsen leg that injects no nemesis (wyrd #257 Iter-1); a day-one test that hand-builds a
healthy_fleetexcluding the killed node (#366 Iter-2); a test drivingMemMetainstead of the real backend (#366 Iter-3).builder.mdalready forbids this in prose but forces no check, so the pattern recurs and the adversary is the only backstop.Change
agents/builder.md.jinjagains a "Before you declare done — refute your own test" section: Do answers three questions and records the answers inbuild-notes.md:Any "no" means not done. The answers are recorded even when all pass, so the human reads them at sign-off.
Single-source:
.claude/agents/builder.md.jinja{% include %}s the body.Verification
cd template && make check— 478 driver unit tests pass.agents/builder.md.jinjaparses cleanly (jinja2).🤖 Generated with Claude Code
Review round 1 (Codex) — addressed in da429fd