fix(security): block all existing-test edits, not just the reproduction - #17
Merged
Conversation
A fix patch could previously weaken any existing test OTHER than the attached reproduction, and in --no-repro mode (used by the UI, watchdog, and loop) no test was protected at all -- so a fix could pass by gutting the very tests it must satisfy (red-team finding). Patch validation now rejects a patch that modifies or deletes ANY pre-existing test file, governed by the new patch.block_test_edits config (on by default; opt out to deliberately edit tests). _is_test_path broadened to cover python/js/ts/go/java/ruby conventions. Roadmap and README guarantee updated to match what the code now enforces. 111 tests pass (+4 covering the cross-test and --no-repro holes and the opt-out).
Noumenon-ai
force-pushed
the
fix/block-all-test-edits
branch
from
June 10, 2026 15:30
93f9166 to
7e7d139
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A fix patch could weaken any existing test other than the attached reproduction, and in
--no-repromode (used by the UI, watchdog, and loop) no test was protected at all -- so a fix could pass by gutting the very tests it must satisfy (red-team finding). This contradicted the roadmap's own guarantee and undermines the reliability-layer thesis.What: patch validation now rejects any patch that modifies/deletes ANY pre-existing test file (not just the reproduction), holding in
--no-repromode. Newpatch.block_test_editsconfig (on by default; opt-out)._is_test_pathbroadened to python/js/ts/go/java/ruby. ROADMAP/README updated to match.Tests: 111 pass (+4: cross-test weakening blocked, --no-repro hole blocked, opt-out allowed, language coverage).