Phase 5c: tracing and metrics — documentation and phase record

[Wahbeh-Mohammad]

Closes #20. Sixth PR of the nine-PR phase-5 stack — the documentation and the phase record for phase 5c's two PRs below it. Targets the tests PR's branch. This is the reconciled tip for phases 5a and 5c together: the rebase onto 5a's stack resolved CLAUDE.md, README.md, docs/README.md, architecture.md, the core README and the roadmap by keeping both lanes' content in 5a-then-5c order with every count re-derived from the combined tree (126 lib files beside version.rb, nine test-mirror exceptions, ten checklists, ten as-built pages), and appended a short factual tail to 5c's roadmap status note recording the rebase — the 4a precedent. 5c's own checklist, design addendum and as-built page are byte-identical to the reviewed tip 3607bfa and describe their own base (730 → 772 manifest rows, nine checklists), as every phase's record does.

What lands

10 files, +970 / −25, all Markdown.

• The checklist, docs/work/mvp/phase5/phase5c/2026-09-09-phase5c-tracing-and-metrics-checklist.md, written from the build: 12 own rows — 11 ✅, each naming its task and the test that proves it; OBS-29 ✅ contract and ordering test only, its unwired half named in the row with its owners (phase 6a Task 9; phase 10's inbound list); OBS-30 ✅ by construction (no rescue in any 5c lib file); OBS-32 ⏳, post-v1 with dexpace-instrumentation-otel (docs/first-release.md's OBS-32/OBS-37 entry) — plus eleven cross-reference rows carrying fourteen IDs (CTX-14/CTX-15, CTX-20, SEAM-28, XCUT-21, OBS-10/OBS-24, OBS-34, OBS-20, XCUT-11, XCUT-20, NFR-11, ASYNC-9/ASYNC-11) the way 4b and 4c carried theirs. Plus what was built, the guards run red (26, eleven on 3.2.11 too), the matrix facts on all four rows, the audit groups run, the deviations from the plan's text (none lowers a gate), the findings routed, and the postponed work.
• The phase 5c design's ledger gains an "As built" addendum, P5-71–P5-76: diagnostics.rb shipped early for 5b to adopt; the Ruby 3.2 floor residual of OBS-23's removal clause; the four doubles namespaced for 5b's plan; Scope.build public @api private; CallableAdapter refusing a non-callable; NO_TRACER#in_span without a block. Numbering follows the phase-5 blocks fixed at dispatch (5a's as-built from P5-51, 5b's from P5-91). docs/sdk-design-ruby/ §8.1 and §10 are frozen; the consolidation and the addenda (the scope handle's identity test, the per-key restore, the floor residual) are a human's, stated in the roadmap note as 3a–4c did. No frozen sentence is contradicted, so no C15.
• docs/knowledge/notes/observability.md gains one entry (append only; sha:manual-phase5c-fiber-nil-and-string-key-floor) narrowing observability/e0f1e864 a third time: Fiber[:k] = nil deletes the key only from Ruby 3.3.0 and retains a nil-valued key on 3.2; Fiber["k"] raises TypeError on 3.2 and 3.3 and interns from 3.4. Both facts had been measured on 3.4.10 alone and stated as facts of the range; 5b's union restore (P5-23) and 8b's ASYNC-9/ASYNC-11 restore are written on them, which is why the roadmap gains a thirty-ninth inbound bullet for phase 10 alongside 5c's status note.
• docs/sdk-documentation/tracing-and-metrics.md — new as-built page (twelve fences, every annotation run verbatim on 4.0.6 and 3.2.11 by implementer and reviewer; the two differ only where the page says the floor does); architecture.md, the core README, README.md and docs/README.md updated to point at it.
• CLAUDE.md — the built-phases sentence gains 5c, the opening paragraph gains the tracing-and-metrics layer, the lib-file count (103 → 109 under lib/dexpace/), the checklist count (nine), and four lines in "Constraints that will bite" (every no-op method works on a frozen receiver; named keywords never a splat on an SPI; Fiber[] per key and never Fiber#storage=; no rescue around a tracer or meter call).

Verification

• bundle exec rake on Ruby 4.0.6 at this tip: exit 0, all seventeen gates green (1,640 / 20,939, 99.97%, YARD 0 undocumented), honest RuboCop 332 files clean.
• ruby .claude/skills/housekeeping/probe.rb: no drift, all eight checks; verify_knowledge_structure.rb OK (2,166 harvested entries, 53 notes, every cited key live); the housekeeping and knowledge suites green.
• Empty diff under docs/product-spec/, docs/sdk-design-ruby/, docs/knowledge/harvested/, docs/deviations.md, docs/first-release.md, every earlier phase's documents, the phase-5 charter, and 5a's and 5b's documents.
• The reviewer re-derived every count CLAUDE.md and the status note stated against 5c's own tree (110 .rb under lib/dexpace/, 110 .rbs mirrors, the six test-mirror exceptions unchanged), read all twelve own rows against their tests (eleven proven, OBS-23 partially — the floor residual, as the row itself says), confirmed the two first-release.md entries and the inbound-list bullet exist where cited, and ran every page fence by hand on both interpreters. The reconciliation agent re-derived the combined counts at this tip, ran the probe (no drift), both verifiers and the full gate set, and ran both as-built pages' fences on 4.0.6.

Known follow-ups from the review (not blocking)

• R0-1 — the checklist's intro, the roadmap note and the implementer's report say "thirteen cross-reference rows"; the table has eleven rows carrying fourteen IDs. Say "eleven rows carrying fourteen IDs" in both places.
• The housekeeping probe's claims check did not flag CLAUDE.md's stale "one hundred and three" lib-file count against 109 on disk before it was rewritten — the spelled-out count appears not to be one the probe derives. The counts here are true by hand-derivation; the check is process tooling, outside any build phase's scope.

[Wahbeh-Mohammad]

Review record for the phase 5c stack (#66 → #67 → #68)

One independent review by a fresh agent with no memory of the implementer, re-running the gates itself on every tip and both interpreters; it approved on the first pass, so no fix round ran. The stack was cut from main at 993c439 (pinned and confirmed unchanged by both agents at start and finish) beside the phase 5a lane, which was cut from the same commit. The reviewed tips were 093e825 → 169a511 → 3607bfa; the pushed tips e1094a8 → d60425f → 032986b are those three commits rebased onto 5a's docs tip d5df0ca by a reconciliation agent — every 5c-only file byte-identical to the reviewed tip, the nine collisions (the entry file, the smoke suite, the regenerated manifest, six prose files) resolved inside the 5c commits, and the three rebased tips re-proven in full (all seventeen gates, the matrix on all four Rubies, 1,849 runs / 56,759 assertions at the tests tip) before the push. Every finding got a stable id, a severity, a file:line and the command output that was its evidence. approve required zero blocking and zero should-fix; the run was capped at four reviews and stopped at the first because it approved.

Round	Verdict	Blocking	Should-fix	Nits	Disposition
0 (final)	approve	0	0	2	carried into the PR bodies

No finding was ever blocking, should-fix, disputed or skipped.

Round 0 (final) — open, carried into the PR bodies

• R0-1 nit — the checklist's intro and the roadmap note say "thirteen cross-reference rows" (the implementer's report says the same); the table holds eleven rows carrying fourteen IDs. Docs branch.
• R0-2 nit — tracing_matrix_facts_test.rb:49 calls Fiber#storage= in one fact-pin test (twice plus the ensure restore), each wrapped in WarningCapture.record, against the brief's letter; no call anywhere in lib/. The reviewer judged no action required; the whole-map writes could move into a subprocess like the fact-6 assertion.

What the reviewer verified, in its own runs

1. Gates — code tip 093e825: all seventeen individually on 4.0.6, all green including the SimpleCov floor (97.86%; the tolerated red the layering rule permits was not needed), the honest RuboCop run 317 files clean; the matrix set on 3.2.11 (98.47%). Tests tip 169a511: full bundle exec rake green on 4.0.6 (1,640 runs / 20,939 assertions, 99.97%), RuboCop 332 files clean, the matrix set on 3.2.11 (100.00% — the registry race branch happened to be covered) and 3.4.10, the core suite on seeds 1, 99991 and 424242 on 4.0.6 and one non-default seed on 3.2.11 with identical run counts. Docs tip 3607bfa: full rake green, probe exit 0 on all eight checks, knowledge verifier OK, all twelve tracing-and-metrics.md fences run on both interpreters (57 annotation checks, 0 mismatches on 4.0.6; on 3.2.11 the only differences are the two the page says the floor has).
2. The handshake in a fresh process, both interpreters — Bundle.members the same eight; Bundle::NONE.span.equal?(NO_SPAN) and .tracer_factory.equal?(NO_TRACER_FACTORY); every #tracer argument shape returns NO_TRACER; NONE's sentinels "0"*32 / "0"*16 and valid? false; every singleton frozen; ObjectSpace shows one no-op class of each kind; #sampled? on "01"/"03" true, "00"/"02" false.
3. The mutation battery — 28 single-edit mutants applied by the reviewer itself (one source edit, two test-file edits, the cop half by source edit, the rest as runtime redefinitions preloaded before the suite), every one caught, eleven on 3.2.11 as well: #recording? true; a mutator returning nil; #finish raising twice; a **attributes splat (caught by Dexpace/NoKeywordSplat and the allocation test); one allocation on a no-op path; the factory minting a fresh tracer; a fresh Scope for the already-current span; no restore on raise; Thread.current[] as the carrier; a prior value not restored; an unset key left present (the 3.2.11 point, run there); a push for a non-recording span; a rescue StandardError inside Tracing; #sampled? reading bit 2; W3C uppercase and all-zero; DATADOG "0" and above 2⁶⁴−1; NONE not the sentinel; a non-nil HTTPTracer default; CallableAdapter forwarding the wrong name; started-after-succeeded and exhausted-without-failed orderings; NO_METER's counter allocating.
4. The layer by experiment, both interpreters — OBS-25 as a two-loop GC.stat delta of exactly 0.0 per call over 22 no-op call shapes at 100/1,000/2,000 iterations with GC disabled (and, without the magic comment, the caller's literals allocating — the lib still nothing); OBS-22/OBS-23 by hand with nested RecordingSpans, restore after a raise, both keys pushed and restored, the previously-unset key absent on 4.0.6 and present-and-nil on 3.2.11 (P5-72, as recorded), inheritance into a child Thread, Fiber and Enumerator with Thread.current[] nil in all three; OBS-27 — 10,000 W3C and 10,000 DATADOG draws all valid and unique, NONE the sentinel by identity, 8 threads × 1,000 concurrent; OBS-28/OBS-29/OBS-30 — NULL answering exactly the eleven, a bare includer, RecordingHTTPTracer's sequence, a raising tracer and a raising meter propagating unwrapped; OBS-31/OBS-33 — one frozen instrument per kind regardless of name, #add(-1), #record(NaN), #record(±Infinity) all nil; Task 11's subprocess with Event, Logger, Configuration and Clock undefined, and with require "dexpace" still undefined (5b absent — the assertion is against the file list); diagnostics.rb defining exactly [:DEFAULT_KEYS, :SPAN_ID, :TRACE_ID], no methods, no requires.
5. The Fiber facts re-run on all four interpreters (plain ruby -W) — Fiber[:k] = nil retains the key with nil on 3.2.11 and deletes on 3.3.12/3.4.10/4.0.6; Fiber["s"] = 1 raises TypeError on 3.2/3.3 and interns from 3.4; Fiber#storage= warns per call (:experimental) on every row and is the only removal on 3.2. The design's fact 1 had been measured on 3.4.10 alone; the reviewer confirmed P5-72's statement and its routing (a new docs/knowledge/notes/observability.md entry, the roadmap's thirty-ninth inbound bullet for 5b's P5-23 and 8b's ASYNC-9/ASYNC-11).
6. Layering — main ⊂ 093e825 ⊂ 169a511 ⊂ 3607bfa, the base unchanged; six commits (four feat:, one test:, one docs:), subjects 55–67 characters, no attribution lines; every file in each diff classified (25 code, 19 tests, 10 docs); lib/dexpace.rb changed by six new require_relative lines only, diagnostics first; no 5a file, no 5b file beyond diagnostics.rb's three constants; async/future.rb, completer.rb, context_store.rb, io.rb, closeable.rb, hooks.rb byte-identical to main; phase 4a's four instrumentation files changed by widening only; sig/ mirrors lib/ 110 == 110; the six test-mirror exceptions unchanged; tools/ untouched; empty diffs under the frozen trees, docs/deviations.md, docs/first-release.md, every earlier phase's documents, the phase-5 charter, 5a's and 5b's documents, and the 5c plan.
7. Global constraints by script over the six new .rb — both header lines; securerandom the only plain require (allowlisted); rescue only in two comments; no ** splat; Thread.current only in a comment; no storage = in lib/; no compact module Dexpace:: form; no ivar write in the three no-op classes; no Enumerator.new; no assert_nothing_raised; GC.stat only inside the allocation assertions; Event, Logger, Configuration and Clock named nowhere in lib/ on this stack.
8. Plan and spec coverage — the 12 tasks' files; each Scope bullet of Phase 5c: Tracing and Metrics #20 mapped; all twelve own rows read against their tests (eleven proven, OBS-23 partially — the floor residual the row itself states); the OBS-32 row's first-release.md entry and the auto-activation entry at their cited lines; --req over the twelve IDs with the appendix-B roll-ups beside the substantive rules, as the charter measured; the ledger addendum P5-71–P5-76 with diagnostics.rb first, the keyword-splat finding verified closed as built, the tracer-factory collision still on phase 10's inbound list, R15's reading stated.
9. Phase record — CLAUDE.md counts re-derived from the docs-tip tree (110 .rb under lib/dexpace/ including version.rb, 110 .rbs mirrors with an identical file list, exactly the six private_constants plus version.rb without a test mirror, nine checklists); the status note's numbers against the checklist; nothing describing 5a or 5b as landed. The brief's expectation that removing the diagnostics line from lib/dexpace.rb yields a LoadError did not hold — scope.rb and tracing.rb require_relative it themselves.
10. Report discrepancies — two coverage lines off by one line (the timing-dependent registry-claim race branch) between the implementer's and the reviewer's 3.2.11 runs; the "rows: 25 / cross_reference_rows: 13" count (R0-1); the 3.3.12 matrix row at the tests tip not re-run by the reviewer (the implementer's claim stands unverified there).

Run: workflow wf_cb2bc327-5ef, 2 agents (1 implementer, 1 reviewer), 1.2M tokens, 2.2 h.