Skip to content

Preserve corpus bytes and prepare bounded observer reads - #288

Merged
rmcdaniel merged 6 commits into
mainfrom
prep/123-buffered-protocol-binding
Nov 6, 2026
Merged

rmcdaniel merged 6 commits into
mainfrom
prep/123-buffered-protocol-binding

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

The startup-observer proposal missed bytes fetched ahead by stock buffering: a one-byte logical read could fetch 8192 raw bytes while reserving only one. This adds a budgeted raw reader and proposed binding for the pinned driver's fresh initial/post-TLS streams. Logical allocation limits, the shared phase budget and deadline, and refusal to discard prefetched bytes remain enforced.

Normal CI also exposed a baseline-reader defect: accepted main's WOFF2 assets crashed corpus validation because Git blob output was decoded as UTF-8. That text path also changed CRLF fixture bytes. The validator now preserves raw Git bytes and resolves the supplied revision once to a validated full commit ID before subsequent reads. Binary assets and unchanged CRLF fixtures have old-fail/new-pass controls, and a symbolic-ref control verifies the exact baseline identity. Corpus selectors, immutability rules and evidence remain unchanged.

Validation: 183 focused observer controls plus 38 accepted portal controls pass locally. All 57 corpus-validator controls pass, and validation against accepted main preserves 43 codec and 104 replay cases. Ruff and focused mypy pass. Current-head hosted checks remain authoritative.

The proposed binding has not imported/composed PyMySQL or opened a socket. Actual driver/TLS/framing, process/backend exit and concrete historical inputs remain unqualified. This does not diagnose the original stalls, change the SDK runtime/API, or admit historical execution. The published portal and its separate status-feed follow-through remain unchanged.

Refs #123 and its source/input decision.

@rmcdaniel rmcdaniel added the agent:sdk-python Work by the sdk-python repo agent label Nov 6, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

The first changed-head run found a concrete CI blocker at job127665530937: _ref_files decoded every base-revision blob through a text subprocess, so accepted main's WOFF2 font bytes raised UnicodeDecodeError before corpus validation. The same text path also normalized CRLF and falsely rejected an unchanged fixture as modified.

Two inert Git controls reproduce both failures on the old reader: non-UTF8 binary asset and byte-identical CRLF fixture. The narrow successor reads Git blob stdout as raw bytes, preserving binary data and exact fixture line endings. It skips no files, narrows no selector and changes no corpus policy/evidence. All 56 validator controls pass, including the two old-fail/new-pass cases. Actual validation against base d3e72974 now passes with unchanged 43 codec /104 replay evidence, no related product change and no invented new fixture credit. Ruff passes. The prepared reader/portal221 focused controls remain separately passed.

The correction stays on this existing PR because the current accepted binary-asset baseline blocks any later PR's corpus check. Fresh exact-head CI is required; the original failure remains failed. It grants no historical reader/SQL/runtime or #184 acceptance and triggers no docs publisher.

@rmcdaniel rmcdaniel changed the title Account for buffered reads in the startup observer proposal Preserve corpus bytes and prepare bounded observer reads Nov 6, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

Current head is 40cf2db4346b9c48e60569d1856d0fbfbde442e2, tree 0a97c6cdd93918fa408c38ca2195db9df2114812. The raw-byte corpus fix passed the hosted corpus job on 96ac9be. Its remaining two Sonar findings are corrected here: Git show ends option parsing before the revision argument, and the buffered-source refusal test constructs its budget outside the exception assertion.

The 56 corpus-validator controls, actual base validation (43 codec /104 replay unchanged), 13 buffered-reader controls and Ruff pass locally after these corrections. Fresh current-head CI and Sonar are running normally. Earlier failed checks remain failed; no policy selector, fixture rule, SDK behavior or publication path was relaxed.

@sonarqubecloud

sonarqubecloud Bot commented Nov 6, 2026

Copy link
Copy Markdown

@rmcdaniel

rmcdaniel commented Nov 6, 2026 •

Copy link
Copy Markdown
Member Author

Exact head 9e30a79ed4840a0a7c4f835252eac8b0f2c47bfe / tree d377f01c57da66647803acd006ab4539e5101297 completes normal source qualification against d3e72974. All 20 current-head checks succeeded, including Python 3.10–3.15, integration, corpus, lint/package, CodeQL, boundary scans, Sonar and the required target-branch qualification. Direct Sonar PR inventory is zero unresolved findings.

The final validator resolves a supplied symbolic ref to a validated complete commit ID before all later baseline reads. Its 57 focused controls preserve the two binary/CRLF old-fail/new-pass cases and verify exact commit binding; the hosted corpus check preserves 43 codec /104 replay cases. Observer/portal controls and the 13 buffered-reader controls remain passed. Original failed/superseded heads retain their outcomes.

This is source acceptance only. Actual PyMySQL composition, TLS/auth, blocked-I/O cancellation, framing and process/backend cleanup remain unqualified. No historical stall diagnosis or #184 operating authority is claimed. The eight changed test/proposal/CI-script paths do not match the docs-push filter, and normal package publication has no main-push trigger. I am proceeding through the protected merge flow, then own actual-main qualification, stable portal preservation and exact branch cleanup.

@rmcdaniel
rmcdaniel merged commit 6f30853 into main Nov 6, 2026
20 checks passed
@rmcdaniel
rmcdaniel deleted the prep/123-buffered-protocol-binding branch November 6, 2026 02:40
@rmcdaniel

Copy link
Copy Markdown
Member Author

Actual main acceptance is complete at 6f30853b3ee1d632b8350b39303092fbfe607d6c / tree d377f01c57da66647803acd006ab4539e5101297, ordered parents [d3e72974,9e30a79e]. The merged tree exactly matches the reviewed head. All 20 PR checks and 18 actual-main checks succeeded. Original main workflows: CI42150990637, Sonar42150990533, CodeQL42150989416, Public Boundary42150990550. Integration and teardown finished successfully; no rerun was needed on the accepted head/main.

The corpus baseline now preserves binary/CRLF bytes and resolves revisions once to a complete commit ID. All 57 validator controls pass; the unchanged 43 codec /104 replay inventory remains enforced. The focused observer and accepted portal controls retain their source results. Prior failed/superseded source checks remain separately attributed.

No docs workflow was triggered by these eight test/proposal/CI-script paths, and no publisher was dispatched. Five ordinary live byte comparisons match the accepted portal artifact: audit, landing, SDK reference, Client API and Worker API. The live audit still binds docs d3e72974 + SDK dc47958c /2.14.1, not the new source main. Real status hydration remains the separate Cloud-owned #285 tail.

The exact merged remote branch prep/123-buffered-protocol-binding was deleted and independently verified absent; its local branch was removed after detaching the clean review worktree at main. The shared main checkout was advanced normally. Frozen #184 sources/inputs and its policy-blocked operating boundary were not changed.

This completes this #123 source slice, not the original startup-stall acceptance. Actual driver composition/import closure, TLS/auth inputs, blocked-I/O cancellation/framing and process/backend exit remain unqualified; all historical collector vectors are unexecuted. General retains later input/execution selection. The next independently selected source cohort is additive clock #215 under 6675221957, with the confirmed Python API shape.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:sdk-python Work by the sdk-python repo agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant