RELEASE-FIX-D: mask the headless User-Agent (F-770) - #47
Open
AminDhouib wants to merge 3 commits into
Open
Conversation
FIX-C shipped on a strongly evidenced hypothesis that was wrong, so FIX-D
measures before it fixes. "The UA leaks" is not precise enough to act on:
a pre-launch --user-agent= flag and a post-launch Emulation.setUserAgentOverride
cover different subsets of the surface, so D0 records each vector separately on
every qualified cell BEFORE D1 picks a mechanism.
V1 navigator.userAgent page
V2 the HTTP User-Agent header the fixture wire -- what the server READ, not
server actually received what the page claims
V3 userAgentData brands + high entropy page
V4 Browser.getVersion() -> userAgent CDP
V2 is new evidence: a page cannot read its own request headers, so the armed
probe issues one same-origin POST to the fixture app's existing /api/echo route,
which reflects the headers it read off the wire. A page-only assertion can pass
while the real header still leaks.
test_f770_ua_vectors_are_measured asserts only that every vector is OBSERVABLE
(a vector that silently stops being collected would make the F-770 signals
vacuous without turning anything red); the leak verdict stays in GATE_SIGNALS
and the strict xfail. Both product and control readings land in the release
artifact, which is also the differential D2 will need.
Also renames the artifact-dir knob off the STEALTH_MCP_* namespace: settings.
_reject_unknown_prefixed_env fails get_settings() for any unknown key there, so
exporting STEALTH_MCP_STEALTH_ARTIFACT_DIR would have detonated the backend the
moment CI used it.
fix-d-stealth-cells.yml is temporary scaffolding: the three-cell offline-stealth
edge lives in W2's release-gate.yml, which is not in this branch's ancestry, and
FIX-D needs the measurement on all three cells. Same selector as W2's edge;
delete when release-gate.yml reaches this history.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
D0's first three-cell run measured Windows/X64 and failed to launch Chrome at
all on Linux/X64 and macOS/ARM64: "Failed to connect to browser", ~3s after the
first spawn. Cause is the lane, not the product. The gate selector
`-m "stealth and not online"` selects test_stealth.py ALONE, so — unlike the
full `-m integration` lane where earlier E2E modules already paid for it —
nobody has absorbed Chrome's cold start, and nodriver gives the debug port only
a few seconds. This is why the same tests are green in W4's ubuntu integration
lane and red here.
Fix is the shared idempotent warmup e2e_helpers already owns for exactly this
("the first Chrome launch on CI is slow / flaky"), not a widened timeout. Linux
also gets Xvfb + DISPLAY, mirroring W2's integration cell.
NOTE for W2/W5: W2's release-gate `offline-stealth` edge runs this same isolated
selector, so it inherits the same cold-start exposure the moment W4's tests land
under it. This commit fixes it at the module, which that edge picks up for free.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… F-774 D0 measured, on all three qualified cells under headless=True, Chrome 150: V1 navigator.userAgent .................. LEAK Linux, Windows, macOS V2 HTTP User-Agent header the server READ LEAK Linux, Windows, macOS V3 userAgentData brands / sec-ch-ua ..... clean Linux, Windows, macOS V4 Browser.getVersion().userAgent ....... LEAK Linux, Windows, macOS Two of the plan's stated assumptions were wrong, which is what D0 exists for: V3 does NOT leak (brands are identical headless and headed everywhere), and V4 IS covered by --user-agent= (the flag is process-wide and Browser.getVersion follows it). So M-A alone covers every leaking vector; M-B is not implemented. --user-agent-product= was measured too and is inert in release Chrome. D1. merge_browser_args now supplies a masked default --user-agent= when the caller chose none. Placement is deliberate: * It is a LAUNCH FLAG, so it is process-wide: every tab, worker and subresource, including tabs the page itself opens, and it reaches the real HTTP request header. A per-target Emulation override would satisfy the spawn tab and leak on the next one -- the failure mode FIX-C's create_hook re-arm existed to fix. * It lives in merge_browser_args because that is the seam W4's vanilla control neutralizes (D2/§4). Masking anywhere else would have masked the CONTROL too, vanilla_detected would have gone false, and the whole stealth suite would have silently become vacuous. The signature is unchanged, so the existing control disables the new behaviour with no edit -- and the differential is asserted, not assumed. * The mask is CONSTRUCTED, not guessed: Chrome's reduced UA froze the platform token and the WebKit/Safari tokens, leaving only <major>.0.0.0 to resolve, so the string is byte-identical to what the same binary emits headed. The three platform tokens are the ones measured on the runners themselves. Version resolution is lru_cached per executable, and Windows reads the version-named sibling directory rather than shelling out (chrome.exe --version hands the flag to a running Chrome instead of printing). Unresolvable version or unknown platform => no mask: a UA that contradicts sec-ch-ua would be a worse tell than the honest headless one. Zero net lines in browser_manager.py, which sits exactly at its LOC cap. D2/D3. ua_no_headless_token moves from XFAIL_SIGNALS into the gating table with three more rows -- the wire header, page-vs-wire equality, and UA-major-vs- client-hints coherence -- and the strict xfail is DELETED, not relaxed. Pinned against a real browser: a tab created later through new_tab, an explicit caller user_agent still winning, V4 at the CDP level, and the differential that the control's UA still contains HeadlessChrome while the product's is that same string minus the token. F-774, opened by this fix and measured not assumed: a --user-agent override makes Chrome blank the high-entropy UA client hints (architecture, bitness, platformVersion, uaFullVersion, fullVersionList). brands/mobile/platform -- and therefore every sec-ch-ua* header actually on the wire -- stay correct, so the residue is JS-only and strictly smaller than the server-side substring test it replaced. Recorded as a strict xfail with the before/after and routed in audit/stage2/finding_F774_ua_client_hints_high_entropy_blanked.md; W5's contract must qualify the claim accordingly. Fixing it needs userAgentMetadata, which is per-target -- a different mechanism, deliberately out of scope. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Member
Author
Final CI evidence
|
| cell | product navigator.userAgent / HTTP header / Browser.getVersion |
control (unmasked) |
|---|---|---|
| Linux/X64 | ... (KHTML, like Gecko) Chrome/150.0.0.0 Safari/537.36 — clean ×3 |
... HeadlessChrome/150.0.0.0 ... — LEAK ×3 |
| Windows/X64 | ... (KHTML, like Gecko) Chrome/150.0.0.0 Safari/537.36 — clean ×3 |
... HeadlessChrome/150.0.0.0 ... — LEAK ×3 |
| macOS/ARM64 | ... (KHTML, like Gecko) Chrome/150.0.0.0 Safari/537.36 — clean ×3 |
... HeadlessChrome/150.0.0.0 ... — LEAK ×3 |
22 passed, 1 xfailed on every cell (the xfail is F-774). The control still leaking on
all three is the proof that the stealth suite did not become vacuous: the masking
sits behind the seam the control neutralizes, and the product UA is exactly the control
UA with HeadlessChrome → Chrome, byte-identical otherwise.
CI (ubuntu)
- quality (ruff / ty / vulture / suppression owners / file budgets): success
- unit tests py3.11 / py3.12 / py3.13: success
- browser integration:
1 failed, 76 passed, 1 xfailed
The one failure is pre-existing on the base branch, not from this PR:
FAILED tests/test_e2e_transport.py::test_real_stdio_release_gate_journey
FileNotFoundError: console launcher 'stealth-chrome-devtools-mcp' not found in
/home/runner/.local/share/uv/python/cpython-3.12.13-linux-x86_64-gnu/bin
It fails identically on audit/release-4-w4 itself (PR #45, run 30132415379): W4's
test.yml integration job installs with uv sync --extra test, and the resolver looks
for the console script beside the interpreter rather than in the project venv.
Untouched here — the base's problem, and W2's release-gate.yml does not have it.
Local gates
ruff format --check + ruff check clean · ty check --exit-zero-on-warning src/stealth_chrome_devtools_mcp/ = 76 diagnostics (baseline, unchanged) · vulture
clean · suppression owners clean · file budgets clean with no cap padded
(browser_manager.py still 1532/1532 — the fix adds zero lines there) · unit lane
764 passed, 1 skipped · integration lane green locally.
RED-first evidence
Disabling only the load-bearing half — merge_browser_args returning combined_args
instead of _apply_default_user_agent(combined_args) — turns exactly four tests red,
each for the right reason:
FAILED test_product_offline_stealth_gate (the 4 new gating UA signals)
FAILED test_f770_a_later_tab_is_covered (new_tab leaks)
FAILED test_f770_product_masks_ua_while_control_still_leaks
assert control_ua.replace(...) == product_ua
- ke Gecko) HeadlessChrome/150.0.0.0 Safari/537.36
+ ke Gecko) Chrome/150.0.0.0 Safari/537.36
FAILED test_product_ua_client_hints_high_entropy_pinned_gap [XPASS(strict)]
That last one is the F-774 xfail self-policing: without the mask the high-entropy hints
are populated, which is direct proof that the mask is what blanks them.
AminDhouib
added a commit
that referenced
this pull request
Jul 29, 2026
…n (F-780) #43, #45 and #47 are red on the legacy `Browser Integration Tests (Chrome + Xvfb)` check while main and #42 are green on the same check. Read at face value that says FIX-B broke browser integration. It did not, and the reason is provable without reading a single CI log. test.yml runs `pytest -m integration --timeout=120`. The harness it drives declares INIT=60, LIST=130, SPAWN=120, WARMUP=150 with 4 attempts -- under a comment saying "the pytest --timeout is the outer net". Two of those single-step bounds are LARGER than the whole job's per-test budget, and BACKEND_READY_TIMEOUT is exactly equal to it. A 120s outer net cannot contain a 150s inner step. The job is structurally incapable of finishing that test; no timing luck is involved. It only started failing at FIX-B because W1 landed the transport journey already xfail'd (B1 was open), and an xfail costs nothing on the clock. FIX-B C2 correctly removed the marker once B1 was fixed, so from that commit the test actually runs -- and immediately hits the wall. The controlled comparison is #46: it contains FIX-B transitively, runs the SAME `-m integration` selection on the SAME OS under the release gate's 180s budget, and is green 23/23. Deliberately NOT fixed. W2 deletes this job, so patching a doomed file on three in-review branches is churn that also mutates PRs under human review. Recorded instead, because the trap is expensive: the merge queue will show red at #43 -> #45 -> #47 until W2 lands, and someone will otherwise go hunting for a regression that was never there. Not added to RELEASE_CONTRACT.md on purpose -- the contract describes what a user receives, not our merge plumbing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes F-770 — under
headless=Truethe product's User-Agent advertised the literaltoken
HeadlessChrome, the cheapest server-side bot check in existence, on a packagewhose headline claim is undetectable automation.
Executes
audit/stage2/plan_RELEASE_FIX_D.md(D0 → D1 → D2 → D3). Stacked onaudit/release-4-w4(PR #45), which ownstests/test_stealth.py. Human holds themerge gate — do not merge.
D0 — the measurement, which changed the plan
RELEASE-FIX-C shipped on a strongly evidenced hypothesis that turned out to be wrong,
so FIX-D measured every vector on every cell before choosing a mechanism. Under
headless=True, Chrome 150:navigator.userAgentUser-Agentheader the fixture server receiveduserAgentData.brands/getHighEntropyValues()/sec-ch-uaBrowser.getVersion() → userAgentTwo of the plan's stated assumptions were wrong — exactly what D0 exists to catch:
UA-CH mechanism (M-B) was never required.
--user-agent=. The plan lists it as not covered; the flag isprocess-wide and
Browser.getVersionfollows it.--user-agent-product=was measured too: inert in release Chrome.V2 is new evidence. A page cannot read its own request headers, so the armed probe
issues one same-origin POST to the fixture app's existing
/api/echoroute, whichreflects the headers it read off the wire. A page-only assertion can pass while the
real header still leaks.
D1 — mechanism: the pre-launch
--user-agent=flag (M-A alone)It is a launch flag, so it is process-wide: every tab, worker and subresource,
including tabs the page itself opens, and it reaches the real HTTP request header. A
per-target
Emulationoverride would satisfy the spawn tab and leak on the next one —the failure mode FIX-C's
create_hookre-arm existed to fix.The mask is constructed, not guessed: Chrome's reduced UA froze the platform token
and the WebKit/Safari tokens, leaving only
<major>.0.0.0to resolve, so the string isbyte-identical to what the same binary emits headed. Version resolution is
lru_cachedper executable; Windows reads the version-named sibling directory rather than shelling
out (
chrome.exe --versionhands the flag to a running Chrome instead of printing).Unresolvable version or unknown platform ⇒ no mask: a UA that contradicts
sec-ch-uais a worse tell than the honest headless one.Zero net lines in
browser_manager.py, which sits exactly at its LOC cap.D2 — the sensitivity control survives, and is asserted
The masking lives in
merge_browser_argsbecause that is the seam W4's vanillacontrol neutralizes. Masking anywhere else would have masked the control too,
vanilla_detectedwould have gone false, and the whole stealth suite would havesilently become vacuous. The signature is unchanged, so the existing control disables
the new behaviour with no edit — and the differential is asserted rather than assumed:
That one equality proves the fix is real, the control is still detected, and the mask
is coherent — same platform token, same major, byte-identical otherwise. The same
differential is asserted on the wire, not just in the page.
D3 — the pin flipped honestly
ua_no_headless_tokenmoved fromXFAIL_SIGNALSinto the gating table with threemore rows (wire header, page-vs-wire equality, UA-major-vs-client-hints coherence), and
the strict xfail was deleted, not relaxed. Also pinned against a real browser: a tab
created later through
new_tab, an explicit calleruser_agentstill winning, V4 atthe CDP level.
What is now claimable — and what is not
Claimable: a default headless spawn no longer advertises
HeadlessChromeon anyvector that reaches a site (page UA, HTTP header, CDP), on all three qualified cells,
asserted as gating signals with no xfail and no skip; the wire client hints stay
coherent with the masked UA; an explicit caller
user_agentalways wins.Not claimable: this closes the UA vector only. W4's other signals and the residual
wall stand unchanged, and the product is not thereby "undetectable". Headed mode never
leaked this token and is unaffected.
New finding opened by this fix — F-774
A
--user-agentoverride makes Chrome blank the high-entropy UA client hints(
architecture,bitness,platformVersion,uaFullVersion,fullVersionList).brands/mobile/platform— and therefore everysec-ch-ua*header actually on thewire — stay correct, so the residue is JS-only and strictly smaller and more expensive
to exploit than the server-side substring test it replaced. It is measured, recorded as
a strict xfail with the before/after, and routed in
audit/stage2/finding_F774_ua_client_hints_high_entropy_blanked.md. W5'sRELEASE_CONTRACT.mdmust qualify the stealth claim accordingly. Fixing it needsEmulation.setUserAgentOverride'suserAgentMetadata, which is per-target — adifferent mechanism, deliberately out of FIX-D's scope.
Two lane defects found and fixed on the way
STEALTH_MCP_STEALTH_ARTIFACT_DIR(W4's artifact-dir knob) is an unknown key inthe product's env namespace, and
settings._reject_unknown_prefixed_envfailsget_settings()for any such key — exporting it would have detonated the backend themoment CI used it. Renamed off that namespace.
-m "stealth and not online"selects
test_stealth.pyalone, so nobody absorbs Chrome's cold start and nodriver'sconnect window expires: the first three-cell run failed to launch Chrome at all on
Linux and macOS while passing on Windows. Fixed with the shared idempotent warmup
e2e_helpersalready owns for exactly this. W2'srelease-gateoffline-stealthedge runs the same isolated selector and inherits the same exposure — it picks this
up for free.
fix-d-stealth-cells.ymlis temporary scaffolding: the three-celloffline-stealthedge lives in W2's
release-gate.yml, which is not in this branch's ancestry. It runsW2's selector verbatim and should be deleted once
release-gate.ymlreaches thishistory.