Skip to content

Phase 7b: server-sent events — the stream layer and the serde-boundary gate - #81

Merged
Wahbeh-Mohammad merged 2 commits into
mainfrom
27-phase-7b-server-sent-events
Sep 20, 2026
Merged

Wahbeh-Mohammad merged 2 commits into
mainfrom
27-phase-7b-server-sent-events

Conversation

@Wahbeh-Mohammad

Copy link
Copy Markdown
Contributor

Part of #27. First PR of phase 7b's three-PR stack — the code — cut from main at c53638b (phases 0–6). Phase 7b is one of four lanes built concurrently off that base (7a #26, 7c #28, 8a #30 are the others); it depends on none of them and none of them is on this branch. Its tests are the next PR up and the phase record the one after.

What lands

The server-sent-events layer under Dexpace::SSE and the workspace's eighteenth blocking gate — 55 files, +1,739 / −8 — SSE-1–SSE-40 (SSE-41, MAY, declined for v1 per docs/first-release.md).

  • Dexpace::SSE::LineReader — the WHATWG line machine over Dexpace::IO::BufferedSource#getbyte and never 3a's #read_line_utf8 (P7-20): the three terminators (\n, \r, \r\n as one), MAX_LINE_BYTES (1 MiB) refused before the line is assembled, over the four-method _ByteSource interface (getbyte, skip, peek, close — the close is what lets the reader release its peek view under strict Steep, P7-82).
  • Dexpace::SSE::Event — the immutable five-field value on the phase-1 pattern (include Model, .build, #with re-validating through .build); retry stays the member name (RBS accepts it; Ruby refuses retry = … and super(retry: retry), so #initialize forwards with a bare super and validates the hint through its reader).
  • Dexpace::SSE::Reader — the field machine: the BOM flag as its one persistent state (skipped once, and only as the first three bytes — xyzdata: and \xEF\xBBxdata: are not a BOM), the retry: screen (SSE-11's four ignore triggers plus a width bound of ten digits before Integer() ever runs, MAX_RETRY_MS = 2,147,483,647), MAX_EVENT_BYTES (8 MiB) counting every line of the block, comments and unknown fields included, checked before the line is applied and reset at every dispatch — dispatching or not (SSE-1; the fieldless-block reset is round 0's R0-1 repair, in afb9faf).
  • Dexpace::SSE::Stream — the resource-owning single-pass facade: .open(response, logger:), .owning(source, resource:, logger:), .borrowing(source, resource:, logger:); Closeable's latch; the quiet-versus-loud release split — an automatic clean-terminal release failure (natural EOF, the typed DONE) is swallowed and reported out-of-band through the existing Dexpace.close_quietly(self, logger:) (5b's route emits http.instrumentation.close; no second emission path, never the payload — P7-83), while an explicit #close propagates (SSE-30); every block-form exit, Enumerable#first(n) included, closes loudly through #drive's ensure; the one rescue ::Exception lives in #drive and #advance has none (P7-86). SSE-31's cross-thread close is proven over BufferedSource.wrapping(IO.pipe) with the reader parked, on every interpreter.
  • Dexpace::SSE::TypedStream — the mapper adapter over Stream#typed (its .new is private): the mapper's value is yielded, SKIP drops the event, DONE ends iteration and closes without yielding (SSE-34); the block form drives Stream#each with no fiber, the external form (#values) drives the raw enumerator — and releases on first(n) / take / find / each { break } exactly as Stream#events does (round 0's R0-2 repair, also in afb9faf).
  • Dexpace::SSE::Sentinel — the two singletons' type, renamed from the design's Signal, which would have shadowed Ruby's core ::Signal inside the namespace on every supported interpreter (P7-81; the plan asserted it "shadows nothing Ruby-owned"). .new/.[] private, no public .build, #with refuses, #pretty_print added; SKIP and DONE are minted in sse.rb.
  • Dexpace::SSE::LimitExceededError (#kind :line / :event, #limit) and Dexpace::SSE::StreamStateError (SSE-26's illegal-state error for a second view — < ::StandardError, include Dexpace::Error; 7c's PAGE-14 error takes the same shape by the manager's decision, closing the argument-family half of phase 10's inbound bullet on the two single-use latches).
  • gates:serde_boundary — SSE-37's mechanised MUST as the eighteenth gate, in the tools/ shape: tools/serde_boundary.rb runs a parsed scan (Prism through phase 0's RequireScan for Ruby, the RBS lexer's tUIDENT tokens for sig/) over lib/dexpace/sse/** and sig/dexpace/sse/**, so the SSE-37 YARD the requirement demands is not a false positive; GUARDED asserts at least one match; page/** is listed PENDING and printed (7c's tree is not on this base — the reconcile pass flips it to GUARDED). Wired into the root Rakefile's DEFAULT_GATES after gates:require_allowlist, test/gates/default_task_test.rb's EXPECTED, and CI's once-per-run gates job (not the matrix); twenty-one fixtures under test/fixtures/gates/serde_boundary/. SSE-38's "no inbound retry" is asserted by test.
  • sig/ mirrors lib/ one file per file; the manifest regenerated once, 1,137 → 1,180 (+43), every row read against the object model (P7-85). Earlier-phase files changed: lib/dexpace.rb (one appended # Phase 7b: block of nine requires), io/typed_reads.rb (the #read_line_utf8 YARD only — SSE-11 → SSE-19, its false "phase 7's SSE machine is the caller" premise replaced), test/dexpace_test.rb's LAYERS table, test/gates/default_task_test.rb (eighteen for seventeen).

Decisions taken in the open, against the plan's text

Ledger rows P7-81–P7-86 in the design's As-built addendum and the checklist's "Deviations from the plan" (36 items). The load-bearing ones beyond those above: the plan has thirteen tasks, not fourteen (the brief's count was wrong; nothing renumbered); its Task 1 would have run mise use -g and was not; its five doubles collapse to three top-level ones (SSEFixtures, ScriptedChunked, FakeByteSource) beside the tree's own FakeResponseBody, FakeChunked and a real Response; the at-scale cap tests run over FakeByteSource (0.3 µs/byte) because the #getbyte path costs ~0.9 µs/byte through BufferedSource (routed to phase 10 with the numbers, not fixed here — the design fixes #getbyte as the primitive); Model.own hands an already deep-frozen list back as it is, so a derived Event may share its parent's list (the observable clause — nothing mutable shared, validation on derivation — holds; the design's SSE-20 sentence is narrowed in the addendum).

Layering

Each tip is green under every gate on its own tree. This branch is green on the SimpleCov floor too — 97.53% on 4.0.6 (2,711 runs, 0 failures) and 97.62% on 3.2.11; the tests PR takes the same tree to 99.98% with 2,931 runs / 68,817 assertions and 0 skips.

Verification

  • Independent review, two rounds by two fresh reviewers with a fix round between: round 0 1 blocking / 2 should-fix / 2 nits — the blocking one a missing test mirror the report had claimed existed; the two should-fixes real code defects (the event-cap byte total not resetting on a fieldless block, so unknown-field keep-alives accumulated into a spurious LimitExceededError; TypedStream#values not releasing on an early exit) — fixed in afb9faf and each proven by the round-1 reviewer's own experiment and a control mutation; round 1 approve, 0 / 0 / 2 nits.
  • All eighteen gates individually at this tip on 4.0.6; the matrix set on 3.2.11; honest RuboCop (--ignore-parent-exclusion, 513 files) clean; probe clean at the docs tip.
  • Mutations: 49 (44 caught) and 58 (54 caught) applied by the two reviewers on 4.0.6 and 3.2.11; every survivor an equivalent mutant (e.g. Data#with calling #initialize on 4.0.6 but not 3.2.11, where the same guard is red).
  • By experiment, both interpreters: the decode trap (data: a\xFFb → a�b, and the retag-less form that would yield caf��); the SSE-31 cross-thread close over a real pipe; the 8 MiB event cap at its real value with RSS measured; every ```ruby block of docs/sdk-documentation/sse.md executed and its `# =>` compared (55/55 on 4.0.6, 54/55 on 3.2.11 — the one the page's preamble documents, the private-`new` message wording); `rbs validate` on the floor for the `retry` member and the recursive `_ByteSource`.

Known follow-ups from the final review (not blocking a gate)

  • R1-2 (nit, code) — sse/reader.rb:216: the ten-digit width bound refuses an 11+-digit zero-padded retry: whose magnitude is in range (00000000005 → unset; 0000000005 → 5). WHATWG parses a digits-only run as base ten and SSE-11's four ignore triggers do not name it; no real server emits it and unset is the conservative outcome. A leading-zero byte scan before the width check closes it; recorded here rather than re-cutting the stack.
  • R1-1 (nit, docs) — sse.md:107 says #with "copies [the data list] again"; as built a derived Event shares its parent's frozen list (the checklist and the addendum say so correctly).
  • The tests branch's first commit message (47331d1) says "fourteen suites"; there are eleven under test/dexpace/sse/ plus sse_test.rb — corrected in the follow-up commit's body, never amended.
  • Findings routed to phase 10's inbound list: BufferedSource.over's enumerator restarts #each after a mid-stream failure (a second read re-delivers the body's first bytes rather than nil or a second raise — on every row; the facade is shielded by SSE-27's closed check); the #getbyte throughput above.

Phase 7b, chapter 13 (SSE-1 through SSE-40; SSE-41 declined for v1).
Nine new files under lib/dexpace/sse/ with sig/ mirrors: the namespace
with its three limits (MAX_LINE_BYTES 1 MiB, MAX_EVENT_BYTES 8 MiB,
MAX_RETRY_MS 2^31-1) and the two frozen Sentinel singletons SKIP and
DONE; LineReader, the WHATWG line machine over BufferedSource#getbyte
with a one-byte pushback, built there and not over #read_line_utf8
because IO-14 keeps a lone CR as content where SSE-2 terminates on it
(P7-20); the immutable five-field Event (Data plus Model); Reader, the
field machine whose one persistent state is the BOM flag; Stream, the
resource-owning single-pass facade over Closeable's latch, built through
.open/.owning/.borrowing with logger: on each; TypedStream over a
caller-supplied mapper; and the two namespaced errors.

The design's Signal is Sentinel: a bare Signal inside module
Dexpace::SSE would shadow Ruby's ::Signal (P7-81). _ByteSource has four
methods, close included, because the reader closes its peek view
(P7-82). SSE-30's swallowed release failure is reported through
Dexpace.close_quietly(self, logger:) as an http.instrumentation.close
diagnostic (P7-83).

gates:serde_boundary is the eighteenth gate: tools/serde_boundary.rb
scans lib/dexpace/sse.rb, lib/dexpace/sse/** and their sig/ mirrors with
a parsed scan (prism for every require spelling and every constant read
or path, the RBS lexer for type names), asserts every guarded glob
matches a file, and prints the pagination layer's globs as PENDING until
7c lands (P7-84). Wired into DEFAULT_GATES, gates:list, the default
task's pin and CI's once-per-run gates job, with fixtures under
test/fixtures/gates/serde_boundary/.

Also: the smoke suite's SSE_LAYER and PhaseSevenLayers case, the surface
manifest regenerated once (1137 to 1180 rows, every row read against
the object model), and the #read_line_utf8 YARD in io/typed_reads.rb
corrected (SSE-11 to SSE-19, and its false premise replaced).
…xits

Review round 0 of phase 7b, findings R0-1 and R0-2.

Reader#dispatch returned nil for a block with no field seen before
reset_block ran, so the block's byte total carried over every blank
line that closed a fieldless block. A run of unknown-field keep-alives,
NUL ids or rejected retries between real events accumulated into
SSE-19's event cap and tore the stream down with a spurious
LimitExceededError(kind: :event) once their sum crossed
max_event_bytes -- at the shipped 8 MiB, after 8 MiB of keep-alives.
SSE-1 makes a blank line the end of a block whether or not it
dispatched, so dispatch now resets the block either way; the Event is
built by a private build_event only when a field was seen, and the
attr_reader's YARD says "reset at every blank line, dispatching or not".

TypedStream#drive_values had the rescue Stream#drive has and not its
ensure, so an Enumerable method that stops early on #values -- first(n),
take, find, each { break } -- left the resource unreleased (closes 0)
where the same call on Stream#events released (closes 1): the raw
enumerator it drives is parked mid-#next and never reaches the Stream's
own ensure. The design names the two shapes as mirrors, and the plan's
own fence drove Stream#drive and inherited its ensure. drive_values now
closes in an ensure of its own: a loud release on the block-form exit,
a no-op on the clean end, a DONE and both failure paths, whose latch has
already flipped. No public name or signature changes; reader.rbs gains
the one private build_event line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core Core HTTP, IO, body, context, encoding: HTTP-* IO-* BODY-* CTX-* UTF-* type:feature New capability or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant