Skip to content

fix(export): block synthetic data from reaching production GCS bucket - #580

Merged
lis186 merged 6 commits into
mainfrom
fix/export-guard-test-leak
Aug 21, 2026
Merged

fix(export): block synthetic data from reaching production GCS bucket#580
lis186 merged 6 commits into
mainfrom
fix/export-guard-test-leak

Conversation

@lis186

@lis186 lis186 commented Aug 21, 2026

Copy link
Copy Markdown
Owner

摘要

全域 export 的 CCXRAY_EXPORT_GCS_BUCKET 會被 npm test 的 e2e 繼承,實測上傳了一筆合成 daily row 到公司 GCS(136→137)。CCXRAY_HOME 只隔離路徑不隔離 bucket。

加了兩層抑制 + 正面啟動訊號 + enumeration test 確保新 launcher 不會靜默洩漏。

Detail

The measured leak (2026-08-21)

test/multi-agent-proxy.e2e.test.js boots a full server, inherits CCXRAY_EXPORT_GCS_BUCKET from the developer's ~/.zshrc, and uploaded one synthetic daily summary (agent_id a91e9944, 3 turns, fake providers claude-sonnet-4/gpt-5.5/grok-4.5) to the company bucket. GCS objects went 136 → 137. The synthetic row was deleted after confirmation.

Two-layer suppression in isExportSuppressed()

1. CCXRAY_EXPORT_DISABLE === '1'        → suppress  (explicit, outranks everything)
2. NODE_TEST_CONTEXT && !_uploader      → suppress  (node --test net, NOT liftable)
else                                    → export
  • Layer 1 is set by every synthetic launcher (6 files + CLAUDE.md smoke command)
  • Layer 2 is automatic for npm test / node --test; !_uploader lets the aggregation suite's injected seam still run
  • An enumeration test (test/export-sync-test-guard.test.js) asserts each known launcher carries the flag — removing it from any one of them turns the suite red (verified per-launcher)

Positive startup signal

startExportSync() now prints [ccxray export] exporter active — bucket <name>, flush every 60min on successful activation. Before this, exporter startup was completely silent — only failures printed. The message shares the same suppression predicate so it cannot announce an exporter that flushExport would refuse to run.

uploaderAtEntry snapshot

_uploader is snapshotted once at flushExport entry. Without this, a test's _setUploader(null) cleanup landing during async index traversal could switch an already-permitted flush to the real GCS uploader.

Codex review

5 rounds, 0 rejected findings. Key moments:

  • R4 blocker: CCXRAY_EXPORT_FORCE (an escape hatch) was checked before the test-runner net, so an inherited ambient FORCE disabled both automatic safeguards → FORCE removed entirely
  • R5 blocker: a third layer inferred "synthetic" from CCXRAY_HOME being set, but ccxray-ops/deploy-stable.sh launches the real hub with CCXRAY_HOME="$HOME/.ccxray" → third layer deleted

Files changed

File What
server/export-sync.js isExportSuppressed(), uploaderAtEntry, startup message
test/export-sync-test-guard.test.js New: 5 tests (layer 1, layer 2, seam passthrough, announcement, launcher enumeration)
test/export-sync.test.js Save/restore ambient CCXRAY_EXPORT_DISABLE in setup/cleanup
CLAUDE.md Smoke command carries CCXRAY_EXPORT_DISABLE=1
scripts/boot-smoke.sh CCXRAY_EXPORT_DISABLE=1
scripts/perf/measure.js CCXRAY_EXPORT_DISABLE: '1' in serverEnv
scripts/generate-screenshot-fixtures.js Printed command carries the flag
test/rebuild-index.browser-harness.e2e.sh export CCXRAY_EXPORT_DISABLE=1
docs/grok-testing.md Documented command carries the flag
docs/site/slides/coscup2026/README.md Demo commands carry the flag
docs/site/slides/coscup2026/tools/shoot-spiral.mjs CCXRAY_EXPORT_DISABLE: '1' in spawn env

Companion change (ccxray-ops, not in this PR)

bq/flush-once.js needs updating to use isExportSuppressed — it currently uses a typeof compatibility check that fails closed (exit 1) on an old checkout. This is the safe direction: the tool refuses rather than uploading. Will commit to ccxray-ops after this merges.

Known gap (accepted)

node test/foo.test.js (without --test) leaves NODE_TEST_CONTEXT unset, and the 22 .test.js files that spawn a listening server do NOT set layer 1 individually. Tagging all 22 was rejected: high maintenance and a new test would silently miss it. Normal paths (npm test, CI, node --test <file>) are covered by layer 2.

Verification

Per docs/verification-principles.md (fail-on-old in a throwaway worktree):

Step Result
Old code + old tests (baseline clean) 28/28 pass
Old code + new tests (must fail) 0/5 pass
New code + new tests 5/5 pass
Full suite 2231/2231 pass

Test plan

  • node --test test/export-sync-test-guard.test.js — 5/5
  • node --test test/export-sync.test.js — 28/28
  • Ambient CCXRAY_EXPORT_DISABLE=1 — guard suite 5/5, aggregation 28/28
  • Ambient CCXRAY_EXPORT_FORCE=1 — guard suite 5/5 (FORCE was removed; env is inert)
  • Full suite — 2231/2231
  • Fail-on-old per-launcher (boot-smoke.sh, measure.js, CLAUDE.md each individually removed → test red)
  • Real boot with bucket set prints "exporter active" (port 5603, isolated home)
  • Leak sealed: same e2e with env in scope → GCS objects unchanged (137→137 after guard, vs 136→137 before)

🤖 Generated with Claude Code

https://claude.ai/code/session_01TaJ4EeuCA6BmvvNdkZteuo

Justin Lee and others added 6 commits August 21, 2026 15:19
Measured 2026-08-21: `test/multi-agent-proxy.e2e.test.js` inherited the
developer's globally-exported `CCXRAY_EXPORT_GCS_BUCKET` and uploaded one
synthetic daily row (3 turns, fake providers) to the company bucket (GCS
objects 136 → 137). `CCXRAY_HOME` isolates storage paths but NOT the bucket.

`isExportSuppressed()` in `flushExport` now blocks uploads on two layers:

1. `CCXRAY_EXPORT_DISABLE=1` — explicit flag, set by every synthetic
   launcher (boot-smoke.sh, measure.js, test/*.sh, CLAUDE.md smoke,
   generate-screenshot-fixtures.js, shoot-spiral.mjs). An enumeration
   test catches a missing flag so new launchers fail the suite instead
   of leaking silently.

2. `NODE_TEST_CONTEXT` — automatic net for `node --test` (what `npm test`
   runs). NOT liftable by any env flag, because the measured leak happened
   inside this context. A non-null `_uploader` (test seam) bypasses it so
   aggregation tests keep working.

`startExportSync()` now prints a positive startup signal naming the bucket,
and stays silent when suppressed — so it cannot announce an exporter that
`flushExport` would refuse to run.

Verification (docs/verification-principles.md):
- old code + new tests: 0/5 pass (all red)
- new code + new tests: 5/5 pass
- old code + old tests: 28/28 pass (clean baseline)
- full suite: 2231/2231 pass

Codex review: 5 rounds, 0 rejected findings. Round 4 blocker (a third
layer that inferred "synthetic" from redirected CCXRAY_HOME) was built
and then deleted after round 5 found it would disable production exports
(ccxray-ops/deploy-stable.sh sets CCXRAY_HOME="$HOME/.ccxray").

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TaJ4EeuCA6BmvvNdkZteuo
…ness, grok smoke

M1: Remove redundant test-end cleanup() calls — afterEach already does it.
    The double-call deleted a restored ambient CCXRAY_EXPORT_DISABLE under
    --test-isolation=none.
M3: setsFlag() for .md files now requires the match inside a code fence,
    so CLAUDE.md's prose description of the flag doesn't satisfy the check.
M4: docs/grok-testing.md live smoke command now carries CCXRAY_EXPORT_DISABLE=1.
    Added to the enumeration test's launcher list.
M2: Rejected — the .test.js gap is documented as an accepted known limitation
    in the PR body.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TaJ4EeuCA6BmvvNdkZteuo
…eanup, partial_day

M1b: Save ambient CCXRAY_EXPORT_DISABLE at module load so the first test's
     afterEach(cleanup) can restore it. The partial_day test (which bypasses
     setup()) also clears it explicitly.
Minor: Removed vestigial CCXRAY_EXPORT_FORCE references from the guard test
       — FORCE was deleted from production in an earlier round.
B1: This commit is signed (git commit -s).

Rejected:
M4b: grok-testing.md has two code-fenced commands, both correctly carry the
     flag. The per-file check intentionally passes if ANY fenced command sets
     it — per-command-line granularity is diminishing returns.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TaJ4EeuCA6BmvvNdkZteuo
Signed-off-by: Justin Lee <justinlee@91app.com>
…Y_EXPORT_DISABLE

The test bypasses setup() and manages its own env, but afterEach(cleanup)
still runs. Without saving _savedFlags.disable first, cleanup deletes the
ambient value and later files under --test-isolation=none lose the safety
flag.

Rejected (not in PR scope):
- ADR 0017 duplicate heading
- internal-headers.js wire-protocol-reference update
- usage.test.js temp dir cleanup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TaJ4EeuCA6BmvvNdkZteuo
Signed-off-by: Justin Lee <justinlee@91app.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TaJ4EeuCA6BmvvNdkZteuo
Signed-off-by: Justin Lee <justinlee@91app.com>
@lis186
lis186 merged commit 5f2ce5d into main Aug 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant