Phase 5c: tracing and metrics — documentation and phase record

CLAUDE.md

@@ -13,9 +13,9 @@ three-state PATCH — solved exactly once, and it deliberately does not compete
 Work here is **spec-driven, not feature-driven**. `docs/product-spec/` is normative: 645 numbered requirements
 across 19 prefixes. Before implementing anything, find the requirement IDs it must satisfy.
 
-**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c and 5a are built; the domain model, the seam layer, the byte-streaming
-layer, the body layer, the execution context, the recovery layer, the stage pipeline and the configuration
-layer are the only domain code.** Six gems exist under `gems/`, every one at `0.0.0`. `dexpace-core` carries the HTTP domain model — `Dexpace::Request`,
+**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c, 5a and 5c are built; the domain model, the seam layer, the byte-streaming
+layer, the body layer, the execution context, the recovery layer, the stage pipeline, the configuration
+layer and the tracing and metrics layer are the only domain code.** Six gems exist under `gems/`, every one at `0.0.0`. `dexpace-core` carries the HTTP domain model — `Dexpace::Request`,
 `Response`, `Headers`, `Status`, `Method`, `Protocol`, `MediaType`, `Query`, `RequestOptions`, `HeaderName`, the
 `HeaderSyntax`, `PercentEncoding` and `URL` function modules, and the construction contract `Dexpace::Model` /
 `Dexpace::Builder` under one error root, `Dexpace::Error`
@@ -75,7 +75,18 @@ pair `Dexpace::HTTPDate.format` / `.parse`, the v4 `Dexpace::UUID.generate`,
 `Dexpace::Retryability.retryable_status?` and the static `Dexpace::BuildInfo` constants — plus two
 wirings into earlier layers: `ContextStore.default` reads `Keys::MAX_TRACKED_CONTEXTS` on its first
 call, and the five readers of `IO::MAX_MATERIALIZED_BYTES` read `Dexpace::IO.max_materialized_bytes`
-per call (`docs/work/mvp/phase5/phase5a/2026-09-09-phase5a-configuration-checklist.md`); every other
+per call (`docs/work/mvp/phase5/phase5a/2026-09-09-phase5a-configuration-checklist.md`) — and the tracing
+and metrics layer, §8.1's tracing half, all of it under `Dexpace::Instrumentation`: the span, tracer
+and tracer-factory protocols phase 4a postponed, given to the three no-op singletons' private classes
+in place (`NO_SPAN`'s seven methods, `NO_TRACER`'s `#start_span` and `#in_span`, the factory's `#tracer`
+unchanged) with `_Span` and `_Tracer` filled; the current-span carrier `Tracing` with `.current_span`,
+`.activate`, `.with_span`, `.correlate` and `.with_correlated_span` over one `Fiber[]` slot; the
+three-ivar scope handle `Scope` and the cached singleton `NO_SCOPE`; `TraceIdFlavour#generate_trace_id`
+and `Bundle#sampled?`; the eleven-method HTTP-tracer vocabulary `HTTPTracer` with §8.1's `NULL` and
+`CallableAdapter`; the metrics SPI `_Meter` / `_Counter` / `_Histogram` with `NO_METER`; and phase
+5b's `Diagnostics::TRACE_ID`, `::SPAN_ID` and `::DEFAULT_KEYS`, shipped early because `Tracing` reads
+the two keys — nothing in phase 5 emits the HTTP-tracer vocabulary, and no logger, event or step exists
+yet (`docs/work/mvp/phase5/phase5c/2026-09-09-phase5c-tracing-and-metrics-checklist.md`); every other
 gem's `lib/` still holds its namespace module and a `VERSION` constant and nothing else. Nothing talks to a
 socket yet. The workspace root
 carries the `Gemfile`, `Rakefile`, `Steepfile`, `rbs_collection.yaml`, `.rubocop.yml`, `.yardopts`, `VERSIONS` and
@@ -528,6 +539,32 @@ Each is one line plus the chapter to read before touching the area.
   the store, and neither does `reset_config!` (`CTX-11`; phase 5a's P5-55). The materialisation ceiling is the
   other way round: `Dexpace::IO.max_materialized_bytes` is read per call, so the live configuration governs
   every materialisation and `IO::MAX_MATERIALIZED_BYTES` is only the default and the fallback (P5-56).
+- **The no-op instrumentation singletons are frozen instances of private classes, and every method on
+  them works on a frozen receiver** — `NO_SPAN`, `NO_TRACER`, `NO_TRACER_FACTORY`, `NO_SCOPE`, `NULL`,
+  `NO_METER` and its two instruments write no ivar and answer a constant or `self` from arguments already
+  on the stack, which is also what `OBS-25`'s "selecting a no-op path MUST NOT allocate per call" wants;
+  the suite asserts it as a two-loop `GC.stat` delta of exactly zero, with only frozen constants, Symbols
+  and Integers crossing the loop, because an inline literal at the call site allocates whether or not the
+  callee does (`test/support/allocation_delta.rb`; phase 5c's design, verified fact 1).
+- **No SPI method in the instrumentation subsystem takes a `**` keyword splat — every attributes parameter
+  is one named optional keyword (`attributes: nil`)** — a splat allocates a `Hash` on every call including
+  one passing nothing, which is the spelling `opentelemetry-api` and every Ruby metrics library uses;
+  the phase-0 cop `Dexpace/NoKeywordSplat` refuses it over every gem's `lib/` (P5-42).
+- **The diagnostic context is written per key with `Fiber[k] = v` and never with `Fiber#storage=`**, which
+  warns on every call at the default level on every supported Ruby; the keys are `Symbol`s because
+  `Fiber["k"]` raises `TypeError` on 3.2 and 3.3 and only interns from 3.4; and **`Fiber[:k] = nil` deletes
+  the key on 3.3 and later but retains it with a `nil` value on the 3.2 floor**, whose whole `Fiber` API
+  is `[]`, `[]=`, `storage` and `storage=` — so `OBS-23`'s "remove it if previously unset" is a removal on
+  3.3+ and a nil-valued key on 3.2, which `Fiber[]` and `OBS-10`'s null-skip both read as absent
+  (P5-49, P5-72; `docs/knowledge/notes/observability.md`). The current-span slot holds one immutable span
+  reference and the previously-active span lives in `Scope`'s ivars on the call stack, never in a stack in
+  fiber storage: copy-on-write protects the slot, not the object in it (R13).
+- **Core wraps no tracer, meter or HTTP-tracer callback in a `rescue`** — `OBS-20` carves those calls out
+  of the log-emission containment, so a throwing tracer propagates and can fail the request (`OBS-30`), and
+  the two block forms in `Tracing` restore through `ensure` regardless; a `rescue` added there is a guard
+  the suite runs red. `Tracing.activate` returns the cached `NO_SCOPE` on an **identity** test — the span is
+  already current — never on the recording flag, which would leave a recording span un-restored under a
+  non-recording one (P5-47).
 
 ## Public API surface
 
@@ -627,13 +664,13 @@ probe compares each against the live tree, and a count written anywhere else in
   and `dexpace-conformance`. Each has a gemspec reading `VERSIONS`, a `sig/` mirroring its `lib/` one file
   per file, a smoke suite, a README, a LICENSE copy and a per-gem `Rakefile`. `dexpace-core`'s `lib/` holds
   the phase-1 HTTP domain model, the phase-2 seam layer, the phase-3a byte-streaming layer, the phase-3b
-  body layer, the phase-4a execution context, the phase-4b recovery layer, the phase-4c stage pipeline and
-  the phase-5a configuration layer — one hundred and twenty phase-1, phase-2, phase-3a, phase-3b, phase-4a,
-  phase-4b, phase-4c and phase-5a files under `lib/dexpace/` beside phase 0's `version.rb`, every one
-  mirrored in `sig/`, and every one of the one hundred and twenty but the nine `private_constant`s
-  `hooks.rb`, `bounded_map.rb`, `context/call_key.rb`, `recovery/ownership.rb`, `pipeline/sync_driver.rb`,
-  `pipeline/async_driver.rb`, `configuration/parsers.rb`, `deep_value.rb` and `proxy/resolution.rb` mirrored
-  in `test/`; every
+  body layer, the phase-4a execution context, the phase-4b recovery layer, the phase-4c stage pipeline,
+  the phase-5a configuration layer and the phase-5c tracing and metrics layer — one hundred and twenty-six
+  phase-1, phase-2, phase-3a, phase-3b, phase-4a, phase-4b, phase-4c, phase-5a and phase-5c files under
+  `lib/dexpace/` beside phase 0's `version.rb`, every one mirrored in `sig/`, and every one of the one
+  hundred and twenty-six but the nine `private_constant`s `hooks.rb`, `bounded_map.rb`,
+  `context/call_key.rb`, `recovery/ownership.rb`, `pipeline/sync_driver.rb`, `pipeline/async_driver.rb`,
+  `configuration/parsers.rb`, `deep_value.rb` and `proxy/resolution.rb` mirrored in `test/`; every
   other
   gem is a phase-0 skeleton whose `lib/` holds the namespace module and a `VERSION` constant and nothing
   else. Every adapter gemspec declares `dexpace-core` and no third-party gem yet
@@ -644,16 +681,16 @@ probe compares each against the live tree, and a count written anywhere else in
   `phase1/`, `phase2/`, `phase3/`, `phase4/`, `phase5/`, `phase6/`, `phase7/`, `phase8/`, `phase9/` and `phase10/`. `phase0/`, `phase1/` and `phase2/` each
   carry that phase's design, plan and checklist; `phase3/` carries its segmentation design,
   `docs/work/mvp/phase3/2026-09-08-phase3-segmentation-design.md`, and two sub-phase directories —
-  `phase3/phase3a/` and `phase3/phase3b/`, each holding that sub-phase's design, plan and checklist — nine
+  `phase3/phase3a/` and `phase3/phase3b/`, each holding that sub-phase's design, plan and checklist — ten
   checklists written so far, each at implementation; `phase4/`
   carries its segmentation design,
   `docs/work/mvp/phase4/2026-09-08-phase4-segmentation-design.md`, and three sub-phase
   directories — `phase4/phase4a/`, `phase4/phase4b/` and `phase4/phase4c/`; each holds that sub-phase's
   design, plan and checklist. `phase5/` carries its segmentation design,
   `docs/work/mvp/phase5/2026-09-09-phase5-segmentation-design.md`, and three sub-phase
-  directories — `phase5/phase5a/`, `phase5/phase5b/` and `phase5/phase5c/`; `phase5a/` holds that
-  sub-phase's design, plan and checklist, and the other two each hold a design and a plan. `phase6/`
-  carries its segmentation design,
+  directories — `phase5/phase5a/`, `phase5/phase5b/` and `phase5/phase5c/`; `phase5a/` and `phase5c/`
+  each hold that sub-phase's design, plan and checklist, and `phase5b/` holds a design and a plan.
+  `phase6/` carries its segmentation design,
   `docs/work/mvp/phase6/2026-09-09-phase6-segmentation-design.md`, and three sub-phase
   directories — `phase6/phase6a/` (retry), `phase6/phase6b/` (redirect) and `phase6/phase6c/`
   (authentication); each holds a design and a plan. Phase 6 is the largest **build** phase in the
@@ -716,5 +753,5 @@ probe compares each against the live tree, and a count written anywhere else in
   and closes or narrows five `docs/first-release.md` lines while publishing nothing: every gem stays
   at `0.0.0`.
   Every checklist but phase 0's, phase 1's, phase 2's, phase 3a's, phase 3b's, phase 4a's, phase 4b's,
-  phase 4c's and phase 5a's is still to be written at execution time.
+  phase 4c's, phase 5a's and phase 5c's is still to be written at execution time.
 - There are 40 harvested topics under `docs/knowledge/harvested/`; the harvest ran here on 2026-09-05.

README.md

@@ -15,7 +15,7 @@ not compete with `faraday` or `httpx` on the easiest way to fetch a JSON endpoin
 
 ## Status
 
-**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c and 5a are built.** Nothing is published. The repository holds six gems under
+**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c, 5a and 5c are built.** Nothing is published. The repository holds six gems under
 `gems/`, every one at `0.0.0`. `dexpace-core` carries the HTTP domain model — the frozen,
 validated wire types every later phase stands on (`docs/sdk-documentation/http.md`) — the seam
 layer: the provider registry, the transport and codec seams, the core-owned async pivot,
@@ -38,7 +38,12 @@ transports, and the adapter that installs a recovery transform
 never-throw typed accessors, builder and process-wide slot, the injectable clock with its
 cancellable wait and the deadline the future gained, the proxy model and its never-raising
 resolver, and the RFC 1123 date, UUID, retryability and build-info utilities
-(`docs/sdk-documentation/configuration.md`); nothing talks to a socket yet. The other five are still skeletons — a namespace, a `VERSION`, a gemspec, a
+(`docs/sdk-documentation/configuration.md`) — and the tracing and metrics layer: the span, tracer
+and tracer-factory protocols behind the three no-op singletons, the current-span carrier and its
+scope handle, log correlation over the two diagnostic-context keys, trace-id generation and the
+sampled bit, the HTTP-tracer vocabulary with its ordering contract, and the no-op meter
+(`docs/sdk-documentation/tracing-and-metrics.md`); nothing emits a trace or a metric yet, and
+nothing talks to a socket yet. The other five are still skeletons — a namespace, a `VERSION`, a gemspec, a
 signature mirror and a smoke suite:
 
 | Gem | Namespace | Runtime dependencies today |

docs/README.md

@@ -102,10 +102,10 @@ a different namespace and the probe does not confuse them.
 Six gems exist under `gems/`, all at `0.0.0` and none published — each a gemspec reading the
 root `VERSIONS` file, a `sig/` mirror and a smoke suite. `dexpace-core` holds phase 1's HTTP
 domain model, phase 2's seam layer, phase 3a's byte-streaming layer, phase 3b's body layer, phase
-4a's execution context, phase 4b's recovery layer, phase 4c's stage pipeline and phase 5a's
-configuration layer; the other five are
+4a's execution context, phase 4b's recovery layer, phase 4c's stage pipeline, phase 5a's
+configuration layer and phase 5c's tracing and metrics layer; the other five are
 phase 0's skeletons, a namespace and a `VERSION`. The as-built documentation lands in
-[`sdk-documentation/`](./sdk-documentation/) as each gem gains code; the nine pages written so far are [`sdk-documentation/quality-gates.md`](./sdk-documentation/quality-gates.md), because the
+[`sdk-documentation/`](./sdk-documentation/) as each gem gains code; the ten pages written so far are [`sdk-documentation/quality-gates.md`](./sdk-documentation/quality-gates.md), because the
 gate set is the thing phase 0 built, [`sdk-documentation/http.md`](./sdk-documentation/http.md),
 because the domain model is the thing phase 1 built,
 [`sdk-documentation/seams.md`](./sdk-documentation/seams.md), because the seam layer is the thing
@@ -119,7 +119,9 @@ the error trail are the things phase 4b built,
 [`sdk-documentation/pipelines.md`](./sdk-documentation/pipelines.md), because the stage pipeline is
 the thing phase 4c built, and
 [`sdk-documentation/configuration.md`](./sdk-documentation/configuration.md), because the
-configuration layer and the clock are the things phase 5a built.
+configuration layer and the clock are the things phase 5a built, and
+[`sdk-documentation/tracing-and-metrics.md`](./sdk-documentation/tracing-and-metrics.md), because
+the tracing and metrics layer is the thing phase 5c built.
 
 ## Keeping this file true
 

docs/knowledge/notes/observability.md

@@ -56,6 +56,41 @@ stable key.
   diagnostic-context carrier and are untouched by this one; they are named by marker rather than by key
   because a backticked key must resolve to a harvested entry, and a note's own key is not one.
   <sub>review · `docs/work/mvp/phase10/2026-09-13-phase10-deviation-reconciliation-and-release-readiness-design.md` · high · sha:manual-phase10-obs29-follow-up-clause</sub>
+- **The per-key fiber-storage API is not uniform across the supported range: `Fiber[:k] = nil` deletes
+  the key on Ruby 3.3 and later and RETAINS it with a `nil` value on 3.2, and `Fiber["k"]` interns a
+  `String` key on 3.4 and later and raises `TypeError` on 3.2 and 3.3.** Narrows `observability/e0f1e864`
+  a third time, on the write side, and is the re-run this file's second `## Superseded` entry
+  (`sha:manual-phase5-fiber-slot-and-key-type`, single-interpreter on 3.4.10) asked for before anything
+  rested on it. Measured 2026-09-17 by phase 5c's implementation on 3.2.11, 3.3.12, 3.4.10 and 4.0.6, one
+  script per interpreter and then as a standing test
+  (`gems/dexpace-core/test/dexpace/instrumentation/tracing_matrix_facts_test.rb`, which pins both
+  boundaries against `RUBY_VERSION` so a patch release that moves either fails the matrix). **(1)** After
+  `Fiber[:k] = "v"; Fiber[:k] = nil`, `Fiber.current.storage.key?(:k)` is `false` on 3.3.12, 3.4.10 and
+  4.0.6 and **`true` on 3.2.11**, where `Fiber.current.storage` reads `{k: nil}`; `Fiber[:k]` reads `nil`
+  on every row, a child fiber and a new thread inherit the nil-valued key on 3.2, and the floor offers no
+  other removal -- its whole `Fiber` API is `[]`, `[]=`, `storage` and `storage=`, and the returned storage
+  is a copy whose mutation changes nothing -- so the one way to remove a key on 3.2 is the warned whole-map
+  setter this file's `## Reference` entry records. Phase 5c's design read "`Fiber[:k] = nil` deletes the
+  key" as a fact of the range (its verified fact 4, the charter's fact 1); it is a fact of 3.3 and later.
+  **(2)** `Fiber["dexpace.probe"] = 1` and the read `Fiber["dexpace.probe"]` both raise
+  `TypeError: wrong argument type String (expected Symbol)` on 3.2.11 and 3.3.12 and intern to
+  `:"dexpace.probe"` on 3.4.10 and 4.0.6, so the entry above's "the per-key setter coerces a `String` key
+  to a `Symbol`" holds from 3.4 only; `Fiber#storage=` refuses a `String` key on every row, as it said.
+  **What follows.** For `OBS-23`, phase 5c's per-key restore stays branchless -- "restore each key to its
+  prior value (or remove it if previously unset)" is one assignment on 3.3+ and, on the floor, a
+  present-and-`nil` key that `Fiber[]` reads identically and that `OBS-10`'s "Keys with null values MUST be
+  skipped" folds identically, which is exactly the state its `P5-49` already argues about from the other
+  direction; the as-built row `P5-72` records the floor half, and no core file calls the warned setter.
+  For `OBS-24`'s union restore (5b's `P5-23`) and `ASYNC-9`/`ASYNC-11`'s pooled-worker restore (8b), the
+  same holds: a per-key restore returns a 3.2 worker to a map of nil-valued keys rather than to an empty
+  map, indistinguishable at every reader that skips nulls and visible only through `Fiber.current.storage`
+  itself; both plans were written on the 3.4.10 fact and are audit work for phase 10 (the roadmap's inbound
+  list, the thirty-ninth bullet). For the key TYPE, this settles `R11` harder than the reconciliation
+  did: a `Symbol` is the one spelling every carrier API accepts on every row, and a frozen-`String` key
+  constant would have raised on the floor at the first `Fiber[]=`, not merely at the whole-map setter.
+  Read-side inheritance, copy-on-write per slot, the warned setter's per-call warning and the
+  `storage = nil` divergence all re-measured as the entries above and below record.
+  <sub>review · `docs/work/mvp/phase5/phase5c/2026-09-09-phase5c-tracing-and-metrics-checklist.md` · high · sha:manual-phase5c-fiber-nil-and-string-key-floor</sub>
 
 ## Reference
 - **A pooled worker inherits the *pool creator's* fiber storage and sees nothing set afterwards, so a