Skip to content

RELEASE W5-prep: real-transport get_cookies success path - #52

Open
AminDhouib wants to merge 1 commit into
audit/release-integrationfrom
audit/release-get-cookies
Open

RELEASE W5-prep: real-transport get_cookies success path#52
AminDhouib wants to merge 1 commit into
audit/release-integrationfrom
audit/release-get-cookies

Conversation

@AminDhouib

Copy link
Copy Markdown
Member

Option (a) — the get_cookies hard block clears

plan_RELEASE §2.5's get_cookies hard block is satisfied by option (a): a real
Chrome + real transport test sets a cookie, retrieves it, and asserts its value.
W5 may claim 94 release-qualified tools; option (b) is not needed.

Evidence node for W5's ledger (verbatim):

tests/test_e2e_transport_cookies.py::test_real_transport_cookie_round_trip

Real cookie value asserted: release_gate_cookie = w5-<uuid4-hex>, freshly generated
per run (e.g. w5-352c36fc058b43bcb67b6eb9e9cde0b7). Uniqueness is load-bearing — a
stale cookie left in a reused profile cannot forge the assertion.

What the node proves

Entirely through tools/call over real stdio JSON-RPC against real headless Chrome:

set_cookie -> get_cookies(urls=[url]) -> ASSERT VALUE -> get_cookies()
  -> document.cookie cross-check -> clear_cookies -> assert gone
  • Both retrieval paths are exercised: Network.getCookies (scoped) and
    Network.getAllCookies (the no-argument default a user actually calls).
  • The document.cookie cross-check proves the cookie really reached the browser,
    rather than get_cookies echoing back what set_cookie was handed.
  • clear_cookies removal is proved by re-reading, not by trusting its return value.
  • Mutation-verified: tampering with the value passed to set_cookie makes the node
    fail on the value assertion with a clear message. The assertion has teeth.

Per-tool status for W5: set_cookie, get_cookies, clear_cookies — all three
release-qualified-success on this node.

Why a dedicated node

§2.5 rules that a representative journey cannot carry a per-tool success claim, and
§2.1 names test_real_stdio_release_gate_journey as exactly that. Folding these
assertions into _canonical_journey would have produced evidence W5 must reject.

The machinery is still the one harness — absolute installed launcher, isolated
HOME/session root, fixture app, stdio client, bounded teardown. stages="cookies" adds
a third declared segment on top of it, alongside "full" and "handshake". No second
journey mechanism and no second fixture, so release_gate_harness's standing rule holds.

Bound worth recording in the ledger: macOS transport is excluded under F-773, so this
node's evidence is Linux/X64 + Windows/X64.

The exemption's reason was false — and why that matters

get_cookies was the sole E2E_EXEMPT name, on the grounds that it "hangs against
real Chrome ... and poisons the tab's CDP connection"
. That reason is seam-specific,
not product-wide
. Measured on this base — same tool, same Chrome, only the seam differs:

Path Result
in-process .fn seam Network.getCookies and getAllCookies both hang (30s, no return); the next call on that tab dies with a 10s CDP timeout. The exemption's symptom was accurate.
real stdio transport + detached backend both retrieval paths return; document.cookie agrees; clear_cookies works; later calls fine.

The transport path is the one users actually have. The tool works; the in-process E2E
suite simply could not reach it. So get_cookies moves from E2E_EXEMPT to
E2E_COVERED (covered by the transport node), and E2E_EXEMPT is now empty — the
set-equality tripwire is 94 covered, 0 exempt, and still passes.

The seam hang is recorded at both call sites, not erased, and is routed as a finding
below rather than fixed — plan_RELEASE is zero-src.

Findings routed, not fixed (zero src/ edits in this PR)

  1. get_cookies is unusable through the in-process .fn seam — hangs indefinitely
    and poisons the tab's CDP connection for all subsequent calls. Only reachable over
    the transport. Worth a tracking id: it constrains how any future E2E test may cover
    this tool, and the connection-poisoning blast radius is larger than the one tool.
  2. Cosmetic type mismatchget_cookies is declared -> list[dict[str, Any]] but
    returns nodriver cdp.network.Cookie dataclasses. pydantic serializes those
    correctly, so the wire shape a user receives is right and nothing is broken. But
    fastmcp's result.data reconstructs them as an opaque [Root()], which is why this
    node asserts on structured_content (the raw wire shape) — a future test written
    against .data would look broken for the wrong reason. Called out in the test
    docstring so nobody loses a debugging cycle to it.

Gates

  • ruff format + check: clean
  • ty check --exit-zero-on-warning src/stealth_chrome_devtools_mcp/: 76 diagnostics
    (matches baseline)
  • vulture, suppression owners, file budgets: clean
  • unit lane (-m "not integration"): 821 passed, 1 skipped
  • W1's canonical journey re-verified green alongside the new node (2 passed), so the
    stages change did not regress W1 or W3's smoke path.
  • Pre-commit and pre-push hooks ran in full; --no-verify not used.

plan_RELEASE §2.5's `get_cookies` hard block clears via **option (a)**: a real
Chrome + real transport test now sets a cookie, retrieves it, and asserts its
value. W5 may claim 94 release-qualified tools.

Evidence node:
  tests/test_e2e_transport_cookies.py::test_real_transport_cookie_round_trip

A dedicated collected node, deliberately: §2.5 rules that a *representative
journey* cannot carry a per-tool success claim, and §2.1 names the canonical
journey as exactly that — so folding these assertions into `_canonical_journey`
would have produced evidence W5 must reject. The machinery is still the one
harness (absolute installed launcher, isolated HOME/session root, fixture app,
stdio tools/call, bounded teardown); `stages="cookies"` selects a third declared
segment on top of it. No second journey mechanism, no second fixture.

The round trip, entirely through tools/call:
  set_cookie -> get_cookies(urls=[url]) -> ASSERT VALUE -> get_cookies()
  -> document.cookie cross-check -> clear_cookies -> assert gone

The value is unique per run so a stale cookie cannot forge the assertion, and
the document.cookie cross-check proves the cookie reached the browser rather
than get_cookies echoing what set_cookie was handed. Verified by mutation:
tampering with the value set makes the node fail on the value assertion.
set_cookie and clear_cookies are proved on the same real path (removal is
proved by re-reading, not by trusting the return value).

Why this was believed impossible, and what actually changed
----------------------------------------------------------
`get_cookies` was the sole `E2E_EXEMPT` name, on the grounds that it "hangs
against real Chrome ... and poisons the tab's CDP connection". That reason is
seam-specific, not product-wide. Measured on this base, same tool, same Chrome:

  * in-process `.fn` seam — Network.getCookies AND getAllCookies both hang
    (30s, no return), and the next call on that tab dies with a 10s CDP
    timeout. The exemption's description of the symptom was accurate.
  * real stdio transport + detached backend — both retrieval paths return,
    document.cookie agrees, clear_cookies works, later calls are fine.

The transport path is the one users actually have, so the tool works; the E2E
suite simply could not reach it. `get_cookies` therefore moves from E2E_EXEMPT
to E2E_COVERED (covered by the transport node) and E2E_EXEMPT is now empty. The
seam hang is recorded at both call sites rather than erased, and is routed as a
finding — not fixed here: plan_RELEASE is zero-`src`.

Also noted for routing, deliberately NOT fixed: `get_cookies` is declared
`-> list[dict[str, Any]]` but returns nodriver `cdp.network.Cookie` dataclasses.
pydantic serializes those correctly, so the wire shape a user receives is right
and this is cosmetic; but fastmcp's `result.data` reconstructs them as an opaque
`[Root()]`, which is why this node asserts on `structured_content`.

Gates: ruff format+check clean; ty 76 diagnostics (baseline); vulture,
suppression owners, file budgets clean; unit lane 821 passed / 1 skipped.
W1's canonical journey re-verified green alongside the new node.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@AminDhouib

Copy link
Copy Markdown
Member Author

CI: green, with one recorded flake (not cookie-related)

Full gate is green — all 31 checks pass, including the aggregate
release-gate / release-gate.

The evidence node passed on both cells that run it, confirmed by name in the logs:

tests/test_e2e_transport_cookies.py::test_real_transport_cookie_round_trip PASSED
  • transport (Linux/X64)2 passed, 905 deselected
  • transport (Windows/X64)2 passed, 905 deselected

(The other passing node in those runs is W1's test_real_stdio_release_gate_journey,
re-verified green alongside the new one.)

Recorded flake — install-smoke (sdist Linux/X64)

First attempt of this run failed; it passed on re-run with no code change. Logging it
rather than quietly re-running, per W8's zero-tolerance flake discipline.

Mechanism — a Chrome cold-spawn failure inside W1's canonical journey, with no cookie
involvement:

_canonical_journey -> spawn_browser
ToolError: Failed to spawn browser: Failed to connect to browser
cold_start_warmup: {"attempted": true, "ok": false, "attempts": 2, ...}

Both warmup attempts failed to launch Chrome at all. Evidence it is environmental rather
than a regression from this PR:

  1. install-smoke (wheel Linux/X64) ran the same journey through the same code in the
    same run and passed — the sdist/wheel split is the artifact, not the code path.
  2. The base (PR [STAGING - DO NOT MERGE] release integration tree for W5 evidence generation #51) passes this cell, and this PR does not change FULL_JOURNEY
    behavior: it was warmup; canonical_journey before and is warmup; canonical_journey
    after — only the branch structure moved to make room for stages="cookies".
  3. release_gate_harness already documents exactly this failure mode:
    WARMUP_ATTEMPTS = 2 # the cold launch intermittently fails outright, not just slowly.

So: a pre-existing cold-launch flake in the install-smoke lane, surfaced here by luck. It
is not introduced by this PR and does not touch the cookie evidence, but it is real
and worth a quarantine/tracking decision on its own — WARMUP_ATTEMPTS = 2 was not enough
to absorb it, which is the actionable part.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant