Skip to content

Phase 9: cross-cutting invariants — the checklist, the audit's verdict and the documentation - #101

Merged
Wahbeh-Mohammad merged 17 commits into
mainfrom
9-phase-9-cross-cutting-invariants-and-conformance-docs
Sep 24, 2026
Merged

Wahbeh-Mohammad merged 17 commits into
mainfrom
9-phase-9-cross-cutting-invariants-and-conformance-docs

Conversation

@Wahbeh-Mohammad

Copy link
Copy Markdown
Contributor

Closes #33. Third PR of phase 9's stack — the phase record, the audit's verdict and the documentation — on top of #100. Phase 9 is unsegmented, so this PR closes the phase.

What lands

  • The checklist — 41 own rows (36 ✅, 2 ✅-in-part, 2 N/A, 1 ⏳ owned elsewhere) plus 11 cross-reference rows, sixty mutations every one run red, the seven audit groups re-run in full, the deviations from the plan, and the findings routed. This is the phase's deliverable: the verdict on XCUT-1–24 and all seventeen NFRs.
  • The design's As-built addendum, rows P9-21–P9-33, beside the design's own P9-1–P9-10; nothing renumbered. It also records four prediction-versus-observation corrections made before the disposition run, because a checklist written against a stale prior is the failure this phase exists to prevent: NFR-7, NFR-10 and NFR-17's cited finding were already green on main, and the "one true positive" the whole gates:bounded_map narrative rested on had been repaired by 8c.
  • docs/first-release.md — the gates:bounded_map blocker ticked and closed by this phase's own run, with no phase-10 repair owed.
  • Six dated phase-10 inbound bullets in the roadmap, plus the phase-9 status note. Phase 9 reports and phase 10 repairs, so every defect found outside dexpace-conformance is routed with its evidence rather than fixed: Registry#resolve's unbounded discovery loop (SEAM-5, XCUT-23); XCUT-12's fiber-scheduler form, unasserted because no first-party suite composes a credential path under a reactor; XCUT-9's two residues gates:cause_walk proves neither absent; 7a's SEAM-21 evidence living inside an assertion body and declared in no ID-keyed map; tools/surface.rb being blind to a private_constant module (NFR-4); and PackagingCase's gem-name-to-constant rule covering none of this port's four exceptions (NFR-15).
  • NFR-13's verdict, which is a real gap and is reported rather than repaired: 307 of 307 shipped .rbs files carry no SPDX header, in all six gems, inside the bundle and inside the built .gem files alike. A RuboCop cop parses Ruby and cannot reach .rbs, so nothing checked them. Already owned by phase 10's design addendum A8 and its inbound row 8 — verified still owned, not re-recorded.
  • docs/sdk-documentation/conformance.md (the four suites with run examples), quality-gates.md (three new rows, renumbered to 21), architecture.md, the gem README, README.md, docs/README.md, CLAUDE.md (the built-phases sentence, the twenty-one gates in every place that states the count, the conformance gem's clause), one new entry in docs/knowledge/notes/cross-cutting-invariants.md (nothing edited in place), and this phase's audit group added to the knowledge-lookup skill's table.

Two late corrections, and who verified them

The last two commits close review 4's findings. 5e24008 corrects the checklist's NFR-14 level cell (MUST → SHOULD; appendix C and the phase's own design table both say SHOULD, re-checked by diffing all 41 level cells against the generated map rather than by eye) and appends the dated correction the phase owed to the pre-existing phase-10 inbound bullet on the async-http client cache, whose two claims — an uncapped Clients#@by_origin, and a gate that "stays red until it lands" — are both false against the tree.

ab9a03d is the maintainer's own commit and was verified by the maintainer rather than by a fresh reviewer — stated here because that is a departure from this lane's discipline. It discharges the same premise in the two other places the fix agent flagged but was scoped out of: the roadmap's narration of the 2026-09-13 routing pass, and docs/first-release.md's opening summary, which still called the blocker red and said phase 10's repair clears it. Both are in material this phase may write, so they are fixes rather than findings. The roadmap's paragraph gets a dated bracketed note rather than a rewrite, because the record of what that pass said is itself the point.

Verification

Docs tip: bundle exec rake (all twenty-one gates) green on 4.0.6 — 4,206 runs / 74,618 assertions / 0 failures / 0 errors / 9 skips / 99.82%, test:gates 129/412, cops:test 193/877, YARD 306 files 100% documented, bundler-audit clean; ruby .claude/skills/housekeeping/probe.rb exit 0, "no drift found"; ruby scripts/verify_knowledge_structure.rb OK; the frozen trees, docs/deviations.md and every other phase's documents untouched. The consolidation of the ledger rows into design §10 is a human's — docs/sdk-design-ruby/ is frozen — and docs/deviations.md waits for phase 10.

Known follow-ups (not blocking a gate)

  • gates:bounded_map's allowlist reason for clients.rb is a dated reading, not a mechanism: the integrity test asserts each allowlisted path still reports, not that its reason still holds, so a refactor removing the drain would keep the gate green. The three XCUT-14 cases in clients_test.rb are what actually pin the cap.
  • XCUT-3 and XCUT-12 still have no committed non-conforming double; the checklist names both as deliberate, and XCUT-12's fiber form is on phase 10's inbound list.
  • YARD emits five [warn] lines on Data.define superclasses and one unknown-tag warning in net_http/request_mapper.rb. Pre-existing, non-fatal.

Wahbeh-Mohammad and others added 17 commits September 23, 2026 21:29
Phase 9's instrument. Four suites beside phase 8a's transport one, over
one shared Runner so the five statuses are decided in one place:
InvariantSuite (28 assertions over all twenty-four XCUT ids, ten
private_constant groups), PackagingSuite (8 over NFR-1, 2, 3, 10, 11,
13, 14 and 15, read from published gem metadata rather than a source
gemspec), CodecSuite (2 portable seam properties, re-derived from the
requirement text rather than lifted out of dexpace-serde-json, whose
files are untouched) and ExecutorSuite (7, SEAM-25's harness half among
them, with ASYNC-3 written so it genuinely fails).

Report gains .merge, #to_h and the MUST-level vacuity blocker: Levels
carries every requirement id's normative level, generated from appendix
C, and a vacuity on a MUST fails the run until an acceptance names that
id with a citation. SharedInstance is XCUT-11's structural predicate,
R8's disjunction, with the declaration supplied by the driver and never
read off the subject. Aggregate gives a whole run one verdict and a
preamble that prints what green does not prove; APPENDIX_B.md is the
61-row coverage map.

Three repository-wide invariant scans become blocking gates over one
RubyVM::AbstractSyntaxTree walker -- gates:cause_walk, gates:bounded_map
and gates:seam_names -- taking the set to twenty-one, all in
DEFAULT_GATES and all in ci.yml's gates job. gates:serde_boundary now
asserts its PENDING list empty, which is the clause SSE-37 handed
forward.
Review round 0, R0-1 and R0-2.

R0-1. Three waits in the phase-9 suites carried no bound, so a subject
that never answers parked the whole run instead of failing an assertion.
ExecutorSuite::Shutdown#check_release opened with an unbounded
`transport.entered.pop`: an executor that REFUSES the post -- ASYNC-2's
saturated queue, or a closed pool -- pushes nothing there and the poster
rescues the refusal, so the run hung and the ensure that frees the gate
and closes the pool was never reached. The entry pop now carries the
assertion's own bound and an expired one is :vacuous with its reason:
with no unit started there is no "blocking task on a worker thread",
which is the antecedent ASYNC-3's sentence opens with, and a MUST-level
vacuity blocks the report anyway.

The two `.each(&:join)` sites take the same treatment against a subject
whose submission or whose shared call never returns -- ExecutorSuite's
concurrent_post and InvariantSuite's drive_threads -- each as a Failure
naming the clause, with the whole thread set sharing one budget so
sixteen stuck threads cost one bound and not sixteen. That is 8a's rule
for this gem, which await_closed_connection's `timeout:` fixed: an
adapter that never releases fails the assertion instead of hanging.

R0-2. PackagingSuite's NFR-14 assertion documented "with no source
supplied it is :vacuous with that reason, never a pass, because 'nobody
told us' is not evidence of a single source" and did not implement it --
with no `versions:` every want was nil, every unit was skipped and the
check passed having proved nothing. It now raises the vacuity the
comment names, and the comment says what a PARTIAL source means too.
Found by writing review round 0's R0-4 test, which drives the shipped
DEFAULT_RESOLVE for the first time: every case in packaging_suite_test.rb
supplies its own `resolve:`, so the lambda a real driver gets had never
run.

RubyGems raises Gem::MissingSpecError for a name it cannot resolve, and
that descends from Gem::LoadError < LoadError < ScriptError -- it is not
a StandardError. So `rescue ::StandardError` let it past, and it escaped
Runner's own bare rescue too, aborting the whole run where the suite's
contract is one :vacuous result naming the absent unit. Measured on
4.0.6: a Suite.run naming an uninstalled gem errored out of the first
assertion instead of reporting eight vacuities.

The rescue now names Gem::LoadError beside StandardError, and the
comment says why the explicit class is load-bearing.
…oute

InvariantSuite::Models#only_closes_what_it_created makes two Checks
against one caller-supplied resource: it was not closed, and it still
works. `Borrowed#usable?` was `!@closed`, the exact negation of the
`closed?` the first Check already reads, so no subject the suite can
build separated the two and replacing the second Check's condition with
a literal `true` left the whole gem's suite green. The code comment and
the phase checklist both claimed the opposite -- that a component
tearing the resource down another way would pass the first alone.

`Borrowed` now carries that other way. `#finish` is the adapter-side
teardown reached without going through `#close` -- `Net::HTTP` spells it
exactly that, a pooled client spells it "retire" -- and `usable?` reads
both flags, so a holder that shuts a borrowed resource down without
calling `#close` passes the first Check and fails the second. The
comment is rewritten to say what the double now does and how the gap
was measured.
Every assertion is driven against a deliberately non-conforming double
before a conforming one makes it pass, which is what proves it can fail
at all; the four suites' own suites are that, plus the runner's status
loop, the report's blocker, the predicate's disjunction and the
aggregate's declaration-keyed map.

Three first-party drivers, one per suite with a subject in this
repository: core's for InvariantSuite (fifteen declared shared instances
and the six factories the suite cannot build), dexpace-serde-json's for
CodecSuite, and dexpace-async-thread's for ExecutorSuite, which waives
ASYNC-3 by id and then runs it unwaived in a second test that requires
the failure, so the waiver cannot outlive the limitation it records.

The three new gates get a fixture workspace under DEXPACE_GATE_ROOT and
gates:serde_boundary's abort branch a RUBYOPT prelude, because neither
had a failing-fixture route at all -- and building the first found that
the three gates opened every file relative to the process's CWD.
Review round 0, R0-1 through R0-5, plus the defect R0-4's test found.

R0-1. Three tests for the three waits that carried no bound, each
driving ONE assertion: a subject that blocks or refuses everywhere is
non-conforming in several ways and these are about the bound. An
executor that refuses the post makes ASYNC-3 :vacuous; one whose #post
blocks fails SEAM-12; a shared instance whose #call never returns fails
XCUT-11's first clause. Each double reports its own parked threads, so
the release joins every one and the thread count is where teardown
expects it.

R0-2. The NFR-14 case now asserts the :vacuous its own name always
claimed, with the report's reason, and a second case covers the partial
source the assertion's comment describes.

R0-3. The interpreter-facts test asserted a warning COUNT of one and was
red on the declared 3.2 floor, whose parser also reports `unused literal
ignored` for the same source. The fact is which $VERBOSE value silences
the class, so the assertion is presence and the comment records the
measurement.

R0-4. PackagingCase::DEFAULT_RESOLVE is public locked surface and ran in
no test, because every case supplied its own `resolve:`. It does now --
and the first run of it found the defect the accompanying code commit
fixes, which is the whole reason a shipped default needs a test.

R0-5. RequirementLevels::ROW's nineteen-prefix restriction was asserted
by nothing: loosening it to `[A-Z]+-\d+` left the map at 645 rows over
this repository's own appendix C. A synthetic table holding an RFC
number and an ISO date in the id column is what discriminates, matching
the case the sibling scanner in tools/appendix_b.rb already carries.
Two shapes from the brief's own minimum mutation list survived round 1,
and both survived because every double in the file emitted exactly one
kind of payload or derived one observation from the other.

ChattyPool logs a non-shutdown event payload and a bare String message
at construction, beside its one shutdown on close. It is conforming, so
the whole report stays green -- and replacing ExecutorCase#shutdowns'
Keys::EVENT match with a bare `entries.count` now reddens SEAM-25,
XCUT-13's second half and XCUT-22's no-shutdown half together, which is
the difference between "one shutdown event" and "one sink write".

TearingSeam tears its borrowed resource down through the teardown
beside `#close` rather than through `#close`, so `closed?` stays false
and only XCUT-22's second Check can report it. Replacing that Check's
condition with a literal `true` now fails this test, where before it
left the whole gem's suite green. The existing overreaching-holder test
gains the message assertion its twin carries.
Review round 2 measured five whole InvariantSuite assertions that could be
neutralised with `rake test:gems` still green: XCUT-1, XCUT-2, XCUT-6,
XCUT-7 and XCUT-10. Nothing in the repository drove a subject that made any
of their clauses fail, so each reported :passed because it could not report
anything else -- the failure mode this phase's own R3 exists to prevent.
The file's header made it worse by naming XCUT-6, XCUT-7 and XCUT-10 among
the IDs it drove.

Thirteen defective stand-in cores, in the shape the file already uses: a
Policy whose cancellation? reads only the outermost error and one that calls
a cancellation retryable (XCUT-1); one matching the cancellation type
exactly, so a subtype escapes, and one telling a timeout apart by its
message (XCUT-2); a capability query reading only the outermost error and
one matching a concrete type (XCUT-6); a configured set ANDed with the baked
classifier and a default set that is not the six RETRY-13 fixes (XCUT-7); a
safety gate that re-sends a bare POST and one taking a failure parameter
(XCUT-10). Two more close the same gap where the reviewer's sweep found only
warning-artefact coverage: a redactor forwarding a URL's userinfo and a
default-ALLOW header list (XCUT-19), and an error-body snapshot that
multiplies its cap (XCUT-24).

Measured: neutralising each of the seven assertions whole and warning-free
-- `return nil if subject` as the body's first line, so no local is orphaned
and NFR-6's fatal-warning hook cannot mask the result -- reddens exactly the
new tests for that assertion, thirteen across the seven. Unmutated the file
is 32 runs and 66 assertions, and the gem's own suite is 275 and 1,089.

The header now names only the IDs the file really drives, and says why
XCUT-3 and XCUT-12 are not among them: a double for either would park a
thread past the assertion's own bound and leak it into the base case's
thread count, so their guards stay the two subject mutations.
Review round 3 swept every phase-9 assertion body one at a time and
found one that no document accounted for: XCUT-18's model-layer
`header_syntax_validation` could be neutralised whole with the entire
`rake test:gems` green, because both of its committed tests pin the
model assertion `:passed` and drive only the call-site one. An audit
phase ships the claim that its assertions discriminate, and an assertion
nothing proves can fail makes that claim false while the report counts
it as evidence.

Two defective stand-in cores close it, in the shape this file already
uses. A permissive validator that refuses nothing reddens the NAME
clauses and every VALUE clause but one. A symmetric validator that
applies the NAME rule to outbound values as well reddens exactly the
HTAB clause, so the asymmetry that IS the requirement -- a name must
reject horizontal tab, an outbound value must accept it -- has a control
of its own rather than resting on the real subject.

Proven by mutation: neutralising `header_syntax_validation` whole
reddens both new tests, and deleting the HTAB-accepted check alone
reddens only the second. The file is 34 runs / 74 assertions unmutated.
The checklist carries forty-one own rows and eleven cross-reference
rows, the forty-five mutations run red with the message each produced,
and the aggregate run's verdict: 43 passed, 1 failed, 0 vacuous, 1
waived. The one failure is NFR-13 -- 307 of 307 shipped .rbs files carry
no SPDX header -- which phase 10 already owns by name and which is
verified still owned rather than re-filed.

The design gains an As-built addendum, P9-21 to P9-33. One
docs/first-release.md blocker closes: gates:bounded_map is green, and
async_http's Clients is a verified false positive of the scan rather
than a silenced defect. Six findings go to phase 10's inbound list, none
repaired. One corpus note corrects an earlier phase-9 entry:
gates:drain_loop was not built, so XCUT-14's drain clause ships with the
behavioural assertion alone.

CLAUDE.md, the two as-built pages, both READMEs and the conformance
gem's README are re-derived from the tree; the knowledge-lookup skill
gains this phase's audit group, which did not exist.
Review round 0, R0-6 through R0-12, plus the two code fixes' half that
lives in the documents.

R0-8. The status note gave 91.09 % as the code tip's 4.0.6 coverage and
quoted the same figure correctly for 3.2.11 one clause earlier. It was
the 3.2.11 number twice. Every proof figure in that paragraph is now
re-measured at this round's tips rather than edited: the code tip is
4,020 runs and 94.67 % (11,688 of 12,346) on 4.0.6 and 90.97 % (11,105
of 12,207) on 3.2.11, the tests tip 4,189 runs and 99.82 %, and the
sentence says outright which row each percentage belongs to, because
that is the mistake it exists to stop repeating. The aggregate run was
reproduced at these tips and is unchanged at 43 / 1 / 0 / 1, so the
audit's verdict stands on this round's code and not only on last
round's.

R0-6. The ledger said forty-five mutations went red where the checklist
said forty-four with one killed uncounted; the checklist was right. The
count is now forty-four at implementation plus seven in this round, and
the killed shape is named in the row rather than folded into a total.

R0-7. CLAUDE.md's exclusion list stopped at phase 8c, so it asserted
that phase 9's checklist -- added by this very stack -- was still to be
written. R0-11: the gem README still called TransportSuite
twenty-eight assertions; it has been thirty-four over twenty-eight
TRANSPORT ids since 8c, measured from the loaded suite.

R0-9. The audit-group row claimed no sleep in any new test. There is
one, deliberate, in a non-conforming double whose whole job is to block
past XCUT-13's half-second bound; the row now names it with its reason
instead of denying it. R0-10: a paragraph sitting between rows 26 and
27 ended the mutation table as GFM reads it, leaving nineteen rows to
render as run-on pipes. It moves below.

R0-12. XCUT-19's row claimed clauses (a)-(d) and appendix C has five;
XCUT-13's named two of three. Both scope-outs were already argued in
the code and unstated in the row a reader checks. Each row now cites
where the missing clause is proven -- 5b's HTTPLogging::DEFAULT for
body logging, and 8.3's ban on arranging an interrupt at all.

Rows 46-52 record this round's seven mutations with the message each
produced. Rows 47 and 48 name what was mutated -- the Check inside the
bounded join, not the call to it, which does not compile under -w --
and drop a failure count that did not reproduce: one, not thirteen.
Plan Task 15 Step 3 asks, in as many words, that a mark which disagrees
with the design's prediction say so and name which prediction was
wrong. The checklist recorded the observations and never the
correction, so NFR-7, NFR-10 and NFR-17 read as transcriptions of a
table that predicted the opposite. A paragraph above the NFR block and
a clause in each of the three rows now name them, with NFR-4's narrower
overturning and NFR-13's and NFR-16's predictions that held; P9-36
carries it in the As-built addendum, which had no row for it either.

P9-37 gives the bounded-map allowlist re-adjudication the ledger row
its six written reasons belonged in. Deviation 4 cited P9-27, which is
the DEXPACE_GATE_ROOT repair and is what deviation 6 cites.

P9-38 and P9-39 record the two guards review round 2 repaired, with
their mutations as rows 53 and 54 and a departures subsection of their
own; the mutation count moves from fifty-one to fifty-three here, in
the design's P9-32 and in the roadmap's Guards paragraph, and says
outright that the pair SURVIVED round 1 rather than being newly thought
of. The As-built band is P9-21-P9-39 in both places that state it.
Round 2 added two tests and three lines of lib/, so every run count and
both coverage denominators moved: the code tip is 11,689 of 12,349 on
4.0.6 and 11,106 of 12,210 on 3.2.11, the tests and docs tips 4,191
runs and 74,584 assertions at 12,327 of 12,349, and the 3.2 row 4,003
and 73,808 at 11,733 of 12,210. The paragraph is re-measured rather
than patched, and it now records the seed and repeat runs it always
implied. The aggregate run over all four suites with the real subjects
reproduces unchanged at 43 / 1 / 0 / 1.
Three documents said something the tree does not.

R6's boundary. The audit's own driver is one ADDED file under
gems/dexpace-core/test/, and no document named it: the design's Module
layout still says "no file in gems/dexpace-core/ at all" while its own P9-9
says "the driver is dexpace-core's own suite", and the checklist's and the
roadmap's boundary sentences enumerated the trees phase 9 changed without
it. All three now name the file and the narrow bend it is -- one added test
file, no existing core file modified, renamed or deleted, git
diff --name-status one path at status A -- and P9-40 records the
contradiction by pointing at both sentences rather than editing either away.

The (b) evidence claim. The checklist asserted over every XCUT row that the
assertion had been driven against a deliberately non-conforming double,
"which is what proves the assertion can fail at all". It was false for seven
rows and, until this round's doubles, for five of them at whole-assertion
scope. The paragraph now names the two rows where (b) is still a hand-run
subject mutation -- XCUT-3 and XCUT-12, whose doubles would leak a parked
thread -- and says what review round 3 added for the other seven. P9-41.

The mutation set. "One mutation per assertion, per gate and per piece of
shared machinery" was untrue as written: XCUT-2, XCUT-6, XCUT-7 and XCUT-10
had no row at all and XCUT-1 only subject rows. The preamble says so, rows
55-61 are the seven whole-assertion neutralisations that close it, and the
count moves from fifty-three to sixty.

Proofs re-measured at these tips: tests and docs 4,204 runs and 74,610
assertions at 99.82 % on 4.0.6 and 3.4.10, the same runs at 12,181 of 12,203
on 3.3.12, and 4,016 / 73,834 / 5 skips at 96.09 % on 3.2.11. The coverage
denominators did not move, because the round added no lib/.
Review round 3's sweep neutralised all forty-five assertion bodies in
the four suites one at a time and left four green. Three were
accounted for -- XCUT-3's and XCUT-12's, which this checklist names,
and a private clause helper whose parent assertion is caught. The
fourth, XCUT-18's model-layer assertion, was named nowhere. The claim
that covered it could not see it because it was counted in ROWS, and
XCUT-18 carries two assertions, so the row was covered on one of them.
The evidence paragraph now counts assertions, names the sweep as what
makes it checkable, and says which of XCUT-18's two assertions each
piece of evidence covers. Ledger row P9-42, deviation 16, and the
mutation table gains rows 62 and 63 -- the total moves to sixty-two in
all four places that state it.

Two counts were wrong in the same direction, coverage being wider than
claimed. The core driver's own factory returns TWENTY shared
instances, not fifteen: eighteen frozen, which conform on frozen?
alone, and two unfrozen that declare exactly what they hold. The
checklist's XCUT-11 row and the as-built conformance page both said
fifteen; both now read off the factory. Design R8's nine is a
different list -- the subjects it names across five phase documents --
and stands.

The stale "not counted among the fifty-three" is sixty-two.
Two false sentences in shipped documents, found by review round 4.

R4-1: the phase-9 checklist stated NFR-14's normative level as MUST.
Appendix C and the phase's own design table both say SHOULD. Only the
Level cell moves; the row's verdict, task numbers and evidence sentence
were already correct. Re-checked mechanically rather than by eye: every
XCUT/NFR level cell in the checklist against tools/requirement_levels.rb's
parse of appendix C -- 41 cells, zero differences, where the same check
named NFR-14 and nothing else before the edit.

R4-2: the pre-existing phase-10 inbound bullet on the async-http client
cache still said Clients#@by_origin is an uncapped per-origin map and
that gates:bounded_map "stays red until it lands". Both halves are false
against the tree: 8c shipped the cap and the gate is green. Appended one
dated bracketed correction in place, deleting and renumbering nothing,
on 7a's precedent for a bullet the tree has falsified -- the as-built
ivar is @by_key over a (reactor, origin) key, #fetch calls #drain inside
the same @mutex.synchronize as the insert, #drain loops back to
MAX_ORIGINS evicting closed reactors first, the gate is green over all
307 files with clients.rb allowlisted as a verified false positive of
the scan, and the docs/first-release.md blocker is closed with no
phase-10 repair owed.
Review round 4's R4-2 corrected the phase-10 inbound bullet itself. The
same premise survived in two places the finding did not name, and both
are in material this phase may write, so they are fixed rather than
reported.

The roadmap's narration of the 2026-09-13 routing pass cited the
uncapped Clients#@by_origin as its example of a bullet whose repair is
expected to be moot, and said such bullets stay unchanged until a green
gates:bounded_map run closes them. That run happened on 2026-09-23, so
the paragraph gains a dated bracketed note in place rather than a
rewrite: the record of what that pass said is itself the point.

docs/first-release.md's opening summary still called the blocker red and
said phase 10's repair clears it. It is ticked at its own entry, closed
by this phase's run against a cap 8c had already shipped, and no repair
is owed.
@Wahbeh-Mohammad Wahbeh-Mohammad added type:test Tests only spec:conformance Conformance-suite behavior, or a gap in the suite itself labels Sep 24, 2026
@Wahbeh-Mohammad

Copy link
Copy Markdown
Contributor Author

Review record for the phase 9 stack (#99 → #100 → #101)

5 independent reviews, each by a fresh agent with no memory of the previous one, each re-running every gate itself on every tip (4.0.6 every gate individually at the code tip and the full rake at the tests and docs tips; the matrix set on 3.2.11, 3.3.12 and 3.4.10) and applying its own mutations on both interpreters, with a fix round by a fresh agent between each. The stack was cut from main at 582e33a.

Round Verdict Blocking Should-fix Nits Mutations (caught) Disposition
0 changes required 0 9 3 34 (32) all 12 addressed in fix round 1
1 changes required 0 3 2 44 (42) all 5 addressed in fix round 2
2 changes required 0 3 0 27 (22) all 3 addressed in fix round 3
3 changes required 0 2 1 23 (19) all 3 addressed in fix round 4
4 (final) changes required 0 2 0 42 (39) carried into the PR bodies

Nothing was skipped.

Round 0 → fixed in round 1

  • R0-1 should-fix — gems/dexpace-conformance/lib/dexpace/conformance/executor_suite/shutdown.rb:88: ExecutorSuite's ASYNC-3 assertion HANGS FOREVER against an executor that refuses work — the one unbounded wait in the shipped suite. Fixed on code + tests (1a6ad15): Already committed by the interrupted fixer; I verified it rather than assuming it. Review round 0's own experiment re-run verbatim against the fixed code — ExecutorSuite's ASYNC-3 assertion driven with an executor whose #post raises Dexpace::ClosedError, on a
  • R0-2 should-fix — gems/dexpace-conformance/lib/dexpace/conformance/packaging_suite/dependencies.rb:97: NFR-14 passes vacuously when the driver names no single source, contradicting the assertion's own documented contract, and its test is named the opposite of what it asserts. Fixed on code + tests (1a6ad15): single_version_source now raises Vacuous, "no single source of truth was named for any unit" when subject.versions is empty, which is what its own comment always promised; the test is renamed to what it asserts and a second case covers the partial source the c
  • R0-3 should-fix — test/gates/phase9_ruby_facts_test.rb:75: The phase's own interpreter-facts test is RED on the declared 3.2 floor. Fixed on tests (9f964f7): The assertion is now presence of the duplicated-key message rather than a count of one. Measured on the declared floor: bundle exec ruby -w -Itest test/gates/phase9_ruby_facts_test.rb on 3.2.11 is 10 runs, 26 assertions, 0 failures (it was 10/23/1, "Expected
  • R0-4 should-fix — gems/dexpace-conformance/lib/dexpace/conformance/packaging_case.rb:32: Surviving mutation: PackagingCase::DEFAULT_RESOLVE's rescue is exercised by no test, and it is the path a real driver uses. Fixed on tests + code (9f964f7): The test drives the shipped DEFAULT_RESOLVE for the first time, and writing it found a second defect the interrupted fixer then repaired: Gem::MissingSpecError descends from Gem::LoadError < LoadError < ScriptError and is NOT a StandardError, so the shipped `r
  • R0-5 should-fix — tools/requirement_levels.rb:27: Surviving mutation: RequirementLevels::ROW's 19-prefix restriction is asserted by nothing. Fixed on tests (9f964f7): requirement_levels_test.rb gains the synthetic-table case the sibling appendix_b_test.rb already carried. Mutation M22 re-run: loosening ROW to [A-Z]+-\d+ is now CAUGHT — the parser admits RFC-7235 and ISO-8601 from the fixture table and the expected {"H
  • R0-6 should-fix — docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-design.md:1103: The design's As-built addendum and the checklist contradict each other on the mutation count. Fixed on docs (a117c42): P9-32 corrected: forty-four re-derived and red at implementation, a forty-fifth killed rather than counted and named in the row, plus seven in this round for fifty-one in all. I verified the arithmetic mechanically rather than by eye — the table holds 53 label
  • R0-7 should-fix — CLAUDE.md:1460: CLAUDE.md's checklist sentence is now false: phase 9's checklist exists and is not in the exclusion list. Fixed on docs (a117c42): CLAUDE.md's exclusion list gains "phase 9's" and now says outright that phase 10's is the only one left, so the sentence stops asserting that this stack's own checklist is unwritten.
  • R0-8 should-fix — docs/work/mvp/2026-09-05-ruby-sdk-v1-roadmap-design.md:5168: The phase-9 status note reports the 3.2.11 coverage figure as the 4.0.6 code-tip figure — contradicted by my run. Fixed on docs (a117c42): Not covered by the recovered stash — this was still open. The whole Proofs paragraph is re-measured at this round's tips rather than edited, because the fix commits moved every figure: code tip 4,020 runs / 74,125 assertions / 7 skips and 94.67 % (11,688 of 12
  • R0-9 should-fix — docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:262: The checklist's audit-group table claims "no sleep in any new test"; one new test double uses Kernel.sleep. Fixed on docs (a117c42): The Minitest-conventions row now says no sleep in the suites' own logic or in any assertion, and names the one deliberate Kernel.sleep with its reason: invariant_suite_test.rb's BlockingCloseSeam#close sleeps BLOCK_SECONDS = 0.6 on its first close, past XCUT-1
  • R0-10 nit — docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:213: The mutation table is broken as Markdown: a prose paragraph between rows 26 and 27 ends the table. Fixed on docs (a117c42): The "killed rather than counted" paragraph moves below the table. Verified mechanically, not by eye: the table is now 55 contiguous lines from its header with no blank line inside it, so rows 27–52 render as rows.
  • R0-11 nit — gems/dexpace-conformance/README.md:9: The gem README still calls TransportSuite twenty-eight assertions; it has been thirty-four since 8c. Fixed on docs (a117c42): Measured from the loaded suite rather than copied: TransportSuite.assertions.size is 34 over 28 distinct TRANSPORT ids plus HTTP-17, HTTP-18, XCUT-18 and PAGE-36. The README said twenty-eight assertions over twenty-two ids; both numbers were 8a's, so both are
  • R0-12 nit — docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:116: Two ✅ rows omit a clause of their requirement instead of naming where it is proven. Fixed on docs (a117c42): Both rows now carry the missing clause and where it is proven, and I checked each against appendix C and against the code comment the row cites. XCUT-19(e) "full request/response BODY logging MUST be OFF by default" is 5b's, proven at HTTPLogging::DEFAULT = NO

Round 1 → fixed in round 2

  • R1-1 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-4/gems/dexpace-conformance/lib/dexpace/conformance/executor_case.rb:125: Surviving mutation: ExecutorCase#shutdowns' event-name match is asserted by nothing, and it is the only observation four ExecutorSuite assertions make. Fixed on tests (e95381b): Confirmed the mutation first: replacing ExecutorCase#shutdowns' entries.count { shutdown?(payload) } with a bare entries.count left executor_suite_test.rb green (15/47/0). The cause is the one the reviewer named -- every double in the file emits only the S
  • R1-2 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-4/gems/dexpace-conformance/lib/dexpace/conformance/invariant_suite/models.rb:76: Surviving mutation: XCUT-22's 'still usable' clause in InvariantSuite cannot fail, and the code comment and the checklist both claim it can. Fixed on code + tests (dd03c35): Took the reviewer's first option -- give Borrowed a second, independent observation -- rather than deleting the Check, because the requirement names two properties and the ExecutorSuite twin proving one of them does not make the InvariantSuite half's claim tru
  • R1-3 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-4/docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:105: The checklist never says the design's prediction was wrong for NFR-7, NFR-10 and NFR-17, which the plan's Task 15 Step 3 requires in as many words. Fixed on docs (7593a36): Added a 'Prediction versus observation' paragraph above the requirement table naming every one of the design's six predicted marks, plus a clause in each of the three rows the reviewer named. NFR-7: the design predicted pending phase 0 plan Task 3's reviewed .
  • R1-4 nit — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-4/docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:26: The checklist states the As-built addendum's band as P9-21-P9-33 twice, and the same document cites P9-34 and P9-35. Fixed on docs (7593a36): Both range sentences (checklist :26 and the Deviations preamble) now read P9-21-P9-39, not P9-35: round 2 adds four addendum rows (P9-36 the prediction corrections, P9-37 the allowlist re-adjudication, P9-38 Borrowed's second route, P9-39 the shutdown-event gu
  • R1-5 nit — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-4/docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:314: The checklist's bounded-map deviation cites P9-27, which is a different ledger row. Fixed on docs (7593a36): Took the second of the reviewer's two options, because the manager's decision 4 asked for the adjudication to be recorded with its reasons and dropping the citation would have left it with no ledger row at all. Added P9-37 -- the re-adjudication against as-bui

Round 2 → fixed in round 3

  • R2-1 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-6/gems/dexpace-conformance/test/dexpace/conformance/invariant_suite_core_test.rb:11: Five InvariantSuite assertions have no committed guard at all: XCUT-1, XCUT-2, XCUT-6, XCUT-7 and XCUT-10 can each be neutralised whole with rake test:gems fully green. Fixed on tests (7132416): Confirmed the finding first with my own whole-assertion neutralisation, then fixed it wider than the finding. THE MEASUREMENT: a mutation harness inserts return nil if subject as the first line of one assertion's body -- subject is always truthy, so every
  • R2-2 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-6/docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:89: The checklist states two universal guard claims that are false for four XCUT rows, and phase 10 reads this document first. Fixed on docs (64208b9): Both false claims corrected, and I did NOT take the reviewer's optimistic reading that R2-1's doubles make them true outright. MEASURED which XCUT ids any committed test drives to :failed: invariant_suite_test.rb drives XCUT-11, 13, 14, 15, 17, 22; invariant_s
  • R2-3 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-6/docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-design.md:936: The one sanctioned file under gems/dexpace-core/ is recorded nowhere: the design's own file list still forbids it, and the checklist's and roadmap's R6 sentences omit it. Fixed on docs (64208b9): Recorded in all four places the reviewer named, and the claim is the stronger one. Design: new ledger row P9-40 in the As-built addendum, which names the two sentences that cannot both hold -- the Module layout's 'no lib/ or sig/ file outside `gems/dexpace

Round 3 → fixed in round 4

  • R3-1 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-8/gems/dexpace-conformance/lib/dexpace/conformance/invariant_suite/header_syntax.rb:34: A third InvariantSuite assertion has no committed non-conforming double: XCUT-18's model-layer header_syntax_validation can be neutralised whole with the full rake test:gems green, and the checklist's (b) sentence names only two exceptions. Fixed on tests (59da489): Confirmed the finding first, then closed it on both layers. MEASURED BEFORE: with return nil if subject inserted as the first line of InvariantSuite::HeaderSyntax#header_syntax_validation, invariant_suite_core_test.rb stayed 32 runs / 66 assertions green a
  • R3-2 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-8/docs/sdk-documentation/conformance.md:472: "the fifteen shared instances it declares" is twenty, in the as-built page and in the checklist's XCUT-11 row. Fixed on docs (5ec0909): Measured independently before changing either sentence: loaded gems/dexpace-core/test/dexpace/cross_cutting_invariants_test.rb and called its own shared_instances factory on 4.0.6 -- TWENTY rows, of which 18 are frozen (11 holding no ivars, SEVEN holding state
  • R3-3 nit — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-8/docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:308: The checklist's "not counted among the fifty-three" is stale against its own "Sixty mutations" ninety lines earlier. Fixed on docs (5ec0909): 'not counted among the fifty-three' is now 'not counted among the sixty-two', the set size after this round's two rows. I re-derived the arithmetic rather than trusting it: the table holds 63 numerically-labelled rows plus a 21b, minus the two placeholders at

Round 4 (final) — open, carried into the PR bodies

  • R4-1 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-10/docs/work/mvp/phase9/2026-09-12-phase9-cross-cutting-invariants-and-conformance-checklist.md:158: The checklist's NFR-14 row states the normative level as MUST; appendix C says SHOULD, and the phase's own design table says SHOULD. MEASURED, by diffing the two level columns rather than by reading. I extracted every | XCUT-n/NFR-n | LEVEL | pair from the checklist (41 rows) and every | XCUT-n/NFR-n | LEVEL | row from docs/product-spec/appendix-c-consolidated-normative-requirement-index.md (41 rows), sorted both and diffed. Exactly ONE row differs:
    < NFR-14,SHOULD (appendix C)
    > NFR-14,MUST (the checklist)
  • R4-2 should-fix — /home/mohammad/Projects/ruby-sdk/.claude/worktrees/wf_c3bc325d-627-10/docs/work/mvp/2026-09-05-ruby-sdk-v1-roadmap-design.md:1897: The roadmap's inbound bullet still says Clients#@by_origin is an uncapped cache and that gates:bounded_map "stays red until it lands"; the manager's binding decision 6 required a dated correction appended in place and none was. MEASURED at the docs tip. git diff main..<docs tip> -- docs/work/mvp/2026-09-05-ruby-sdk-v1-roadmap-design.md has exactly TWO hunks: @@ -2761,6 +2761,52 @@ (six NEW inbound bullets appended) and @@ -5044,3 +5090,174 @@ (the phase-9 status note). The pre-existing bullet at :1897 -- which IS inside phase 10's inbound list, the list starting at :1797 -- is untouched. grep -n 2026-09-23 over t

What the final reviewer verified by experiment, both interpreters

  • Whole-assertion neutralisation sweep, WIDER than round 3's: insert return nil if subject as the first line of each of the 57 def X(subject) bodies across InvariantSuite, PackagingSuite, CodecSuite — 57 applied, 50 CAUGHT, 7 SURVIVED (baseline 186 runs / 493 assertions / 0 failures). I then grepped every survivor's name against the four suites' ROWS tables: exactly TWO are assertions -- interruptible_waits (XCUT-3)
  • Are the five surviving clause helpers inert, or guarded by something other than whole-neutralisation? Cut a SUBJECT mutation for two of them and trace the third — None is inert. (a) XCUT-17 clause (c): changing 6b's target.userinfo = "" to = nil in gems/dexpace-core/lib/dexpace/redirect/location.rb reddens the core driver with the clause named -- 'XCUT-17: userinfo in the Loca
  • Reproduce the phase's headline aggregate verdict independently: all four suites with the real subjects, loading BOTH first-party drivers' own factory methods (the core driver's suite_arguments plus Ne — EXACT MATCH with the checklist: 43 passed, 1 failed, 0 vacuous, 1 waived, 0 errored; passed? false; unknown_ids []. Per suite InvariantSuite 28/28, PackagingSuite 7/8, CodecSuite 2/2, ExecutorSuite 6/7. Rendered `waived
  • Design P9-2's claim: build all six gems with gem build -o into a scratch dir, install them with gem install --install-dir (plus json 3.0.2 and the async closure from the bundle cache), and run Pac — 7 passed, 1 failed (NFR-13), 0 vacuous, 0 waived -- the checklist's figure, reproduced against PUBLISHED metadata. core full_gem_path inside the scratch install, runtime_dependencies [], 222 .rbs shipped inside the insta
  • gates:bounded_map over the real tree with the allowlist EMPTIED, to check the six adjudicated entries are exactly the right six — Nine offences in seven files, exactly as AS_BUILT 5 predicts: recording_span.rb:37, challenges.rb:94/:145/:209, bounded_map.rb:46, configuration/builder.rb:40/:44, instrumentation/event.rb:220 and async_http/clients.rb:5
  • Read gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/clients.rb to adjudicate the allowlist entry rather than trusting the reason beside it — CONFIRMED a false positive of the scan. The ivar is @by_key (:56); #fetch does client = (@by_key[key] ||= candidate); drain(evicted) inside ONE @mutex.synchronize; #drain is evict_closed_reactors(evicted) then `evict
  • Interpreter-sensitive: run test/gates/phase9_ruby_facts_test.rb with and without -w on both ends of the range, and reduce AstScan::SYMBOL_TYPES to one node type on each end — Under -w: 10 runs / 26 assertions / 0 failures on 3.2.11 and 3.4.10 alike. Without -w: 10 / 22 / 1 SKIP -- the warning test skips rather than going red, which is better than the brief predicted; SuiteRunner appends -w, s
  • SharedInstance.audit against the three real subjects the plan's filed predicate would have condemned, plus both branches of R8's disjunction — Redactor::DEFAULT (frozen, [:@Policy]) PASS; Logger::NULL (frozen, six ivars) PASS; ContextStore.default (unfrozen, [:@Map] declared) PASS. An unfrozen object with an undeclared ivar raises Failure; a frozen object holdi
  • Flakiness: run the three timing-sensitive suites ten times each under timeout 120 — core/cross_cutting_invariants_test.rb 31/41/0/0/1 ten times identically; conformance/executor_suite_test.rb 15/47/0/0/0 ten times; async_thread/conformance_test.rb 10/21/0/0/1 ten times. No variation, no hang, no leaked
  • Run every phase-9 ruby fence of docs/sdk-documentation/conformance.md verbatim on BOTH interpreters the page claims — All outputs byte-identical to the page and identical on both rows: the four suites' assertion counts [["InvariantSuite",28],["PackagingSuite",8],["CodecSuite",2],["ExecutorSuite",7]]; the first assertion [["XCUT-15"],"pu
  • Re-derive CLAUDE.md's changed count sentences from the docs tip's tree rather than trusting the probe — 220 core lib .rb excluding version.rb (matches 'two hundred and twenty') and 220 sig mirrors; 54 conformance lib .rb excluding version.rb (matches 'fifty-four'); 25 of those privatise their own top-level constant (matche
  • Surface manifest: derive the added rows and check nothing private leaked — +80 rows for dexpace-conformance only (100 -> 180); the other five manifests byte-identical to main. Grepping the manifest for every group module -- Models, Taxonomy, Classification, Memory, Security, HeaderSyntax, Crede
  • Check the checklist's per-ID level column against appendix C mechanically rather than by reading — 41 rows on each side, sorted and diffed. Exactly one differs: NFR-14 is SHOULD in appendix C:648 and MUST in the checklist. Everything else matches, including XCUT-12 and XCUT-24 as the two XCUT SHOULDs and NFR-1/8/10/17
  • Trace the manager's decision-6 correction through the roadmap — The roadmap diff has exactly two hunks -- six NEW inbound bullets at :2761 and the phase-9 status note at :5090 -- and grep -n 2026-09-23 over the whole file hits only those. The pre-existing inbound bullet at :1897 is
  • Knowledge corpus: run this phase's newly added audit-group row and the structure verifier — --gaps XCUT,NFR -> '0 of 41 IDs in 2 prefixes have no substantive entry (0 roll-up only, 0 uncited)'; --prefix-info XCUT -> '24 of 24 IDs have a substantive entry'. The row IS present in .claude/skills/knowledge-look

Tips reviewed at the final round: code dd03c35, tests 59da489, docs 5ec0909 on main 582e33a.


After the fifth review

The table's last row says round 4's two findings were "carried into the PR bodies". They were not — both were fixed, and the record should say by whom.

Round 3 was the fourth review, which is where this lane's rule sends the stack up whatever the verdict. It did not, because R3-1 was a real defect of exactly the kind an audit phase exists to prevent: a systematic sweep of all 46 assertion bodies in the four new suites found four that could be neutralised whole with the suite green, and one of them — XCUT-18's header_syntax_validation — was named in no document, so the phase's own claim that its assertions discriminate was false where it counted that assertion as evidence. That earned the targeted round 4 (fix and review, scoped to review 3's three findings and the re-proof, nothing else), which fixed all three and verified each by evidence it produced itself.

Round 4 then surfaced two more, both documentation and both false sentences in shipped documents: the checklist stating NFR-14's level as MUST where appendix C and the phase's own design table say SHOULD, and a binding decision of the maintainer's that had not been carried out — the dated correction owed to the pre-existing phase-10 inbound bullet on the async-http client cache, whose claims (an uncapped Clients#@by_origin, a gate that "stays red until it lands") are both false against the tree.

Two more commits close them:

  • 5e24008 — a fix agent, scoped to those two. The level cell was re-checked mechanically, by diffing all 41 level cells against tools/requirement_levels.rb's parse of appendix C rather than by eye: 41 cells, zero differences, where the same check named NFR-14 and nothing else before the edit. Every clause of the inbound-bullet correction was verified against clients.rb and a green gates:bounded_map run rather than copied from the wording it was given.
  • ab9a03d — the maintainer's own commit, verified by the maintainer and not by a fresh reviewer. Stated plainly because it is a departure from this lane's discipline. It discharges the same premise in the two places the fix agent flagged but was scoped out of: the roadmap's narration of the 2026-09-13 routing pass, and docs/first-release.md's opening summary. Both are in material this phase may write, so under the repository's own rule they are fixes rather than findings — and leaving them would have recreated the very defect round 4 reported, in the document phase 10 reads first. Verified by the full twenty-one-gate run at the final docs tip on 4.0.6 (4,206 runs / 0 failures / 9 skips / 99.82%), the probe, the knowledge verifier, the frozen-tree diff and the linearity checks.

Nothing was skipped, and the leftovers in the PR bodies are leftovers by judgement, not by exhaustion: the gates:bounded_map allowlist reason for clients.rb being a dated reading rather than a mechanism, XCUT-3 and XCUT-12 having no committed non-conforming double (both named as deliberate in the checklist, XCUT-12's fiber form on phase 10's inbound list), and pre-existing non-fatal YARD warnings.

@Wahbeh-Mohammad
Wahbeh-Mohammad changed the base branch from 9-phase-9-cross-cutting-invariants-and-conformance-tests to main September 24, 2026 20:57
@Wahbeh-Mohammad
Wahbeh-Mohammad merged commit b242de6 into main Sep 24, 2026
2 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

spec:conformance Conformance-suite behavior, or a gap in the suite itself type:test Tests only

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Phase 9: Cross-Cutting Invariants and Conformance

2 participants