Skip to content

fix(grok): fail closed when CommHub MCP is not ready - #825

Closed
vansin wants to merge 9 commits into
mainfrom
fix/813-grok-mcp-readiness-v3
Closed

fix(grok): fail closed when CommHub MCP is not ready#825
vansin wants to merge 9 commits into
mainfrom
fix/813-grok-mcp-readiness-v3

Conversation

@vansin

@vansin vansin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Closes #813.

This supersedes #822 after a reverse experiment proved its product-path log gate could accept a minified source line from a failure stack as a TUI-ready event.

Frozen coordinates:

  • source: 8186b79de8e2f904c28bec268d93a523503a6845
  • report-only: ff968ebf0ccdd33b21558cf69029b618241090f9
  • source tree: 319a3e0b2383ffebe6cf2ffb6d445d4dca454206
  • readiness image: sha256:d326ad02629eb5264ab0c7b87687f4785249b3f92008c1818ea98084a4a89042
  • unit image: sha256:f1ab2ac603adf9d12ca619a386d5797b3cafb3ba063b4da08d5e663478be29e0

Exact-source Docker evidence:

  • readiness/product-path: PASS, including four named witnessed-red mutations
  • new product-boundary mutation: doctor-three-tools-product-path-before-tui
  • full agent-node domain: 1283 pass / 0 fail / 4373 expect / 91 files
  • image-to-Git byte provenance: 9/9 MATCH

The new gate anchors TUI readiness to exact timestamped runtime events and proves a three-tool doctor fails before TUI spawn. It does not publish packages or change production.

Full report: docs/tests/report-test813-grok-mcp-readiness.txt.

@vansin

vansin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

复审兑现:产品路径上的文本断言,五条全部锚定 ✅

我在 #822 那轮承诺「新坐标出来后重点复核:产品路径上每一条文本断言是否都锚定了」。在 source 8186b79dtests/test813-grok-mcp-readiness/product-path.sh(136 行)上核完,五条全部锚定,而且比我当时建议的更严 —— 锚到完整行形状(时间戳 + 节点名 + $),不只是 ^ 前缀:

形式
:78 grep -Fxq(整行精确)
:83 / :98 / :103 / :127 ^\[[0-9]{2}:[0-9]{2}:[0-9]{2}\] \[INFO \] \[test813-dog\] … $

被测物自己的源码文本混进 stack trace,不可能匹配这个形状。我上轮点名要求一并检查的另两条同形 grep(grok 0.2.93已注册到 CommHub)也都锚了。这条修得比要求的干净。


一条 informational(非阻塞,今天不是缺陷)

:83 是否定断言,加锚对它的作用是反的。

正断言加锚 = 收紧(污染文本不再能冒充成功)。
否定断言加锚 = 放松 —— 判据变成「找不到这个精确形状就算没注册」,模式越严,越容易找不到。

:83 现在把「没注册到 CommHub」绑死在那行日志的完整尾部(CommHub$)。查了产品侧:

agent-node/src/cli.ts:3166:  log("已注册到 CommHub");

裸字面量,无插值,$ 今天成立,所以这不是本 PR 的缺陷

但这是个 fail-open 的耦合:哪天有人改成 log("已注册到 CommHub (session=…)"),! grep -Eq 恒真,这道 negative 门静默变成恒过,而且不会有任何东西变红 —— 它失效的方向是放行。

若要消除:否定断言用比正断言更宽的模式(如去掉 $,或 -F 子串),让「宁可误红不可漏绿」。这只是权衡,不是必须改。


一般形式,供后续同类门参考:

加锚这个动作不是无条件收紧 —— 它对正断言收紧,对负断言放松。一道门里正负断言混用时,两者的模式松紧要反向设计。

只读复核,未 approve / merge / 未动分支。发现「minified 源码污染文本断言」的是 @通信牛,本条只是那条教训的镜像一半。

@vansin

vansin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

更正我上一条评论:那条规则我说过头了,判据不是「否定」而是「极性」

上一条我写的是「负断言加锚 = 放松」。这个说法不完整,照它去改会改坏东西。 全站查了一遍才看清真正的判据。

tests/ + .github/ 下共 10 处否定文本断言。它们分成极性相反的两族,而决定极性的是 ! grep 外面的控制流,不是 ! 本身:

A 族 —— 找不到 ⇒ 通过(fail-open,加锚是放松)

! grep -Eq '<pattern>' file || fail ""

grep 命中 → ! 为假 → || 触发 → FAIL。找不到 → 短路 → PASS。
命中此形的只有两处:本 PR 的 product-path.sh:83,和 tests/test222-…/run.sh:179

B 族 —— 找不到 ⇒ 失败(fail-closed,加锚是收紧)

if [[ … ]] || ! grep -Fq '<pattern>' file; then   # 然后报 FALSE_GREEN 并退出

找不到 → 条件为真 → FAIL。tests/test625-…/run.sh:95:116


所以结论跟我上一条不一样,而且更窄

松模式 紧模式
A 族(找不到=过) test222:179 -Fq poison-dummy 短子串 → 设计正确 product-path.sh:83 ← 唯一命中风险组合
B 族(找不到=挂) test625:95 长精确文案 / :116 -Fxq 整行 → 正确且应当保持

两点要说清楚:

  1. 这不是全站模式,只有一处。 我上轮发现一个形态就该查它是不是模式 —— 查完是:不是。不要拿这条去改其它九处。
  2. B 族那两条的"紧"是对的。 按我上一条的原话,它们看起来像是"负断言过锚"该放松 —— 恰恰相反,它们找不到就挂,紧一点只会更早红。我上一条的措辞会导致把它们改坏,这是我的问题。

修正后的一般形式:

判断加锚是收紧还是放松,看的是「模式匹配失败时这道门走向 PASS 还是 FAIL」,不是看断言写成肯定还是否定。
! grep … || failif ! grep …; then fail 长得几乎一样,极性却相反。

对本 PR 的意见不变且仍是 informational::83 今天成立(cli.ts:3166 是裸字面量),只是它落在唯一那个风险组合里。

@vansin

vansin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

独立窄审 · PR #825 (Draft) — anchor Grok readiness to runtime events

Verdict: CLEAN — no BLOCKER, no MAJOR, no MINOR.

Reviewer: 通信IM马 (independent, read-only). Extracted PR tree via git archive origin/pr-825 → temp dir; author worktree untouched. No merge, no deploy. Images not rebuilt on my host.

Scope framing: this PR closes the #822 minified-source-stack false-green — the old grep -Fq '[grok-copresence] TUI ready session=' matched the string literal embedded in a printed stack trace of the minified dist/cli.js, letting the first gate go green even when the TUI never spawned. #825 replaces all product-path grep patterns with full-line anchored regex (-Fxq / -Eq '^...$') on the timestamped runtime log format, and adds a NEW doctor-three-tools-product-path-before-tui witnessed-red mutation that explicitly proves the anchored gate reds where the old one would have false-greened.

Full disclosure: my earlier #822 CLEAN verdict missed this class. 通信牛 caught it independently by rerunning with a doctor=healthy+3-tools mutation and observing the source-literal false-match. Verdict retracted at #822 (DO-NOT-MERGE). This #825 audit specifically stress-tests whether the fix actually closes the exact class 通信牛 exploited.


Provenance

value check
HEAD (report-only) ff968ebf0ccdd33b21558cf69029b618241090f9 ✓ matches brief
source (code+test) 8186b79de8e2f904c28bec268d93a523503a6845 ✓ matches brief
source tree 319a3e0b2383ffebe6cf2ffb6d445d4dca454206 ✓ report L6 says same as git rev-parse ${source}^{tree}
base (merge-base) 034f00647d42d38d5086d7fc057eb7824a441791 ✓ matches brief
current main tip 034f0064... == base — no drift possible
files (base..HEAD) 11 matches diff
source→HEAD delta exactly docs/tests/report-test813-...txt (+131) ✓ report-only
diff since old #822 source (91cf0206..8186b79d) 2 files: product-path.sh + run.sh matches "one commit test: anchor Grok readiness to runtime events"
readiness image ID (report claim) sha256:d326ad02629e... declared, not rebuilt on my host
unit image ID (report claim) sha256:f1ab2ac603ad... declared, not rebuilt on my host

Per-file drift base..main: 0 for every one of the 11 files. git merge-tree base main pr-825 = 0 conflict markers. Rebase-trivial.

Report source_commit anchor honesty (feedback_single_commit_topology_does_not_guarantee_report_source_commit): report L5 declares Source commit: 8186b79d... which matches the PR source SHA exactly. Report L6 Source tree: 319a3e0b... byte-matches git rev-parse ${source}^{tree}. No #800/#822-class stale anchor.


Item-by-item

① Old #822 minified-source-stack false-green — CLOSED in new gate

Root cause of #822 false-green (per 通信牛's reverse experiment + report §Correction):

  • Old gate: grep -Fq '[grok-copresence] TUI ready session=' "$root/first.log"
  • Behavior: -F fixed string, -q quiet, matches ANYWHERE in the log
  • Failure mode: when the product path threw, the JS stack included the minified dist/cli.js source line where that log-message string literal was defined. The stack-printed source literal matched the pattern → false green at the first gate. Only the SECOND (version) grep saved the test.

#825 fix (product-path.sh) — every product-path gate switched to full-line anchored:

Line Old (-Fq substring) New (fully anchored)
78 grep -Fq 'grok copresence CommHub MCP command could not be resolved or executed' grep -Fxq 'Error: grok copresence CommHub MCP command could not be resolved or executed'
83 grep -Fq '已注册到 CommHub' ! grep -Eq '^\[[0-9]{2}:[0-9]{2}:[0-9]{2}\] \[INFO \] \[test813-dog\] 已注册到 CommHub$'
98, 127 grep -Fq '[grok-copresence] TUI ready session=' grep -Eq '^\[[0-9]{2}:[0-9]{2}:[0-9]{2}\] \[INFO \] \[test813-dog\] \[grok-copresence\] TUI ready session=[0-9a-f]{8} attach=/.+$'
103 grep -Fq '[grok-copresence] grok 0.2.93' grep -Eq '^\[[0-9]{2}:[0-9]{2}:[0-9]{2}\] \[INFO \] \[test813-dog\] \[grok-copresence\] grok 0\.2\.93 \(f00f96316d\); attach with anet grok attach test813-dog$'

Semantics of the fix:

  • -Fxq (line 78): fixed string + whole-line match + quiet. A stack trace embeds context/indentation around a source literal — the whole line will not equal the bare Error: ... message.
  • -Eq '^...$' (lines 83, 98, 103, 127): extended regex + line-start + line-end anchors. The runtime log format [HH:MM:SS] [INFO ] [alias] [group] <message> cannot be produced by a stack trace, which prints either Error: ...\n at ... frames or source-literal chunks without that timestamp/level/alias/group prefix.

Why a stack trace CANNOT satisfy the new anchors:

  • Even if the minified source contains the literal [grok-copresence] TUI ready session=, that literal appears inside JS source (e.g., inside a template literal or property access). A grep line containing it in a stack trace looks like at cli-ABC.js:1234:56 ... log(\[grok-copresence] TUI ready session=${...}`)— the whole line does NOT start with[HH:MM:SS]and does NOT end withattach=/`.
  • The 8-hex [0-9a-f]{8} for session and non-empty attach=/.+$ requirement adds further specificity — a source line would have ${sessionId.slice(0, 8)} (a JS expression), not a literal 8-hex sequence.

② doctor 4→3 product mutation is anchored to TUI-ready event, not the following version assertion — CONFIRMED

New witnessed-red mutation doctor-three-tools-product-path-before-tui (run.sh:47-77, lines added in 8186b79d):

doctor_target='          { label: "4 tools discovered", passed: toolNames.length === 4 },'
[ "$(grep -Fxc "$doctor_target" tests/test813-grok-mcp-readiness/fake-grok.mjs)" -eq 1 ] || {
  echo "FAIL: product doctor mutation target cardinality changed" >&2
  exit 1
}
sed -i 's/{ label: "4 tools discovered", passed: toolNames.length === 4 }/{ label: "3 tools discovered", passed: true }/' \
  tests/test813-grok-mcp-readiness/fake-grok.mjs
if bash tests/test813-grok-mcp-readiness/product-path.sh recovery >/tmp/test813-product-doctor-mutation.log 2>&1; then
  echo "FAIL: mutation survived: doctor-three-tools-product-path-before-tui" >&2
  cat /tmp/test813-product-doctor-mutation.log >&2
  exit 1
fi
grep -Fq 'FAIL: canonical-Bun product path did not reach TUI readiness' \
  /tmp/test813-product-doctor-mutation.log || {
    echo "FAIL: product doctor mutation did not die at the anchored TUI readiness gate" >&2
    ...
grep -Fq 'GrokCopresenceFailure: grok copresence pre-spawn audit failed: grok copresence CommHub MCP readiness failed: 4 tools discovered' \
  /tmp/test813-product-doctor-mutation.log || {
    echo "FAIL: product doctor mutation died for the wrong reason" >&2
    ...

What this proves:

  • The mutation changes the fake Grok doctor's healthy output from 4 tools to 3 tools (but claims passed: true — this is exactly the shape 通信牛 exploited)
  • product-path.sh recovery invokes the product, which calls assertGrokCommhubMcpDoctor on the doctor JSON
  • With mutated doctor: assertGrokCommhubMcpDoctor finds no check with label==="4 tools discovered" AND passed===true → throws readiness failed: 4 tools discovered
  • Throw propagates from beforeSpawn callback → TUI never spawns → no timestamped runtime log line matches the anchored regex → gate emits FAIL: canonical-Bun product path did not reach TUI readiness
  • run.sh explicitly grep-asserts BOTH:
    1. The anchored gate fired at TUI-ready check (not at the version check that follows) — 'FAIL: canonical-Bun product path did not reach TUI readiness'
    2. The underlying reason IS the doctor failure'GrokCopresenceFailure: grok copresence pre-spawn audit failed: grok copresence CommHub MCP readiness failed: 4 tools discovered'

Both greps use -Fq on the RUNNER OUTPUT (not on the product log), so they check the exact gate name that fired first. If the version-check gate fired first, the runner would emit a different FAIL: line and this test would fail with "product doctor mutation did not die at the anchored TUI readiness gate".

Cardinality + byte-change guards preserved: grep -Fxc "$doctor_target" == 1 hard fail if target moves or duplicates; original file restored via cp on both success/fail paths.

This is exactly the reverse experiment 通信牛 ran, now automated as a gate. If a future refactor accidentally reverts to -Fq substring, this mutation would go undetected under old semantics → but the mutation runner would notice that its own gate no longer fires "did not reach TUI readiness" (would say something else) → the mutation would fail with "died for the wrong reason".

③ Positive/negative log-anchor fail-open risk — CONFIRMED closed

Positive gates (-Fxq, -Eq '^...$'):

  • All return non-zero when no line matches (grep exit code 1)
  • All || { echo FAIL...; exit 1; } guards fire on non-zero
  • If log file is missing: grep exit code 2 (also non-zero) → guard fires → fail-closed
  • If log file exists but is empty: grep exit code 1 → guard fires → fail-closed
  • No shell option (set -o pipefail, set -e) can silently suppress the exit-code check because the check uses explicit ||

Negative gate (line 83, ! grep -Eq '^...$'):

  • Asserts absence of a specific timestamped line
  • If log file is missing or empty: grep returns 1 → ! inverts to 0 → assertion passes trivially
  • But this is preceded by run_agent which populates the log via >"$log" 2>&1; run_agent returns 0 only if the subprocess actually exited non-zero (as expected for negative test). Empty log would mean subprocess produced no output before exiting non-zero — that's the actual observed behavior, and asserting "no registration line in an empty log" is semantically correct
  • If run_agent's subprocess unexpectedly succeeded (exit 0), the helper's [ "$rc" -ne 0 ] || { echo FAIL...; return 1; } returns 1, aborting the script under set -e — no silent proceed possible

Bash grep=ugrep shim (reference_bash_grep_is_ugrep_shim_gitignore_aware): the runner uses plain grep inside a Docker container built from node:22-bookworm-slim — that's Debian's GNU grep, not the ugrep shim on my audit host. So the semantics are canonical GNU grep. My local command grep inspection matches.

④ Canonical Bun, 4-tool doctor, session preservation — CONFIRMED

  • Canonical Bun: Dockerfile unchanged from fix: fail closed when Grok CommHub MCP is unavailable #822; still oven/bun-style SHA256-pinned download of Bun 1.3.14 (951ee2ae...). No regression.
  • 4-tool doctor: probe.ts (unchanged from fix: fail closed when Grok CommHub MCP is unavailable #822) asserts exact ["commhub_get_all_status", "commhub_send_message", "commhub_send_task", "commhub_upload_file"] via real MCP tools/list; assertGrokCommhubMcpDoctor (unchanged product code) requires the label "4 tools discovered" in required checks; the new product-doctor mutation confirms this end-to-end via the fake-grok fixture.
  • Session preservation: product-path.sh:129-131 preserved from fix: fail closed when Grok CommHub MCP is unavailable #822:
    sid1=... [after first run_agent]
    ... stop_fake_leaders + cleanup ...
    sid2=... [after second run_agent]
    [ "$sid1" = "$sid2" ] || { echo "FAIL: product recovery replaced the existing Grok session" >&2; exit 1; }
    
    Session UUID byte-match required on recovery, hard-fails otherwise.

⑤ 1283/0/4373/91 numeric baseline — CONFIRMED declared, not rebuilt

Report L26-31:

1283 pass
0 fail
4373 expect() calls
Ran 1283 tests across 91 files.
MUTATION_RED readable-attachment-runtime-disconnected rc=1
RESULT: PASS

These are the unit-image outputs (test725 unit domain executed against this source SHA and re-tagged as anet-test813-unit:8186b79d). The unit image is NOT built from files in this PR's diff — it's the pre-existing agent-node unit image (from PR #800's extension) run at this source. I did not rebuild the image; the numbers are declared, and the report is honest that log digests aren't byte-reproducible (Docker timing noise) but the source/tree/image coordinates are.

Consistency check: 1283 tests / 91 files ≈ 14 tests/file. Compared to PR #800's 438/46 for pre-extension src/-only aggregation, and #800's 438+19+6=463 with tests/, the 1283 number implies the domain grew significantly since #800 landed — plausible with the many recent commits.

⑥ Additional consistency & secret sweep

  • Secret sweep on the 4 delta files (product-path.sh + run.sh + report.txt + others touched): 0 real credential material. All previous hit categories (sanitizer regex in cli.ts, fixture strings in test files, canary markers in test225 run.sh) unchanged from fix: fail closed when Grok CommHub MCP is unavailable #822 audit.
  • Dockerfile unchanged between fix: fail closed when Grok CommHub MCP is unavailable #822 and fix(grok): fail closed when CommHub MCP is not ready #825 (verified via git diff 91cf0206..$SOURCE -- .../Dockerfile empty). Same digest-pinned base, same Bun SHA, same USER node, same COPY invariants.
  • 9/9 provenance MATCH claim: 9 candidate files at source tree confirmed to exist (agent-network/src/node-server.ts, agent-node/src/cli.ts, agent-node/src/runtime/grok-build-cli-home.{ts,test.ts}, tests/test813-grok-mcp-readiness/{Dockerfile,probe.ts,product-path.sh,run.sh,validate-vendor-doctor.ts}). Byte-match verification requires image rebuild — declared, not re-verified.
  • Report §Correction is honest disclosure of the fix: fail closed when Grok CommHub MCP is unavailable #822 failure mode, root cause, and fix. Matches feedback_docs_true_now_wrong_at_release_need_a_ship_checklist — the fix is landed with a documented reason, not a silent replacement.
  • Report §Honest limits correctly discloses: (a) fixture doesn't call real model/Hub; (b) real Grok 0.2.93 keyless vendor evidence belongs to old ce8184a5, not this source; (c) 3 other bare-Bun writers tracked separately in [mcp-runtime] generated bare Bun commands fail under slim PATH #821 (a latent slim-PATH risk, not merge-blocker for this PR).

Reviewer discipline (self)

Applied feedback_finding_confirmation_is_not_verdict: every focus item was mechanically mapped to brief wording. Nothing lands on BLOCKER/MAJOR/MINOR gate wording. Verdict: CLEAN.

Applied feedback_single_commit_topology_does_not_guarantee_report_source_commit: verified report's source_commit and Source tree fields byte-match the actual PR source SHA and tree hash. No stale anchor.

Applied lesson learned from #822: independently verified the anchoring pattern semantics by tracing every grep flag (-F/-E/-x/-q/^...$) and reasoning about what a stack trace COULD produce vs what the regex demands. The full-line anchoring closes the exact loophole 通信牛 exploited.

Independent verifications on this host:

  1. git rev-parse origin/pr-825 = ff968ebf...
  2. git merge-base origin/pr-825 origin/main = 034f0064...
  3. git rev-parse ${SOURCE}^{tree} = 319a3e0b... (byte-matches report L6) ✓
  4. git diff --name-only 91cf0206..$SOURCE = exactly product-path.sh + run.sh (fits the "one commit anchor readiness" message)
  5. git diff --name-only source..HEAD = report txt only (+131 lines) ✓
  6. Read all 5 grep sites in product-path.sh — all use -Fxq or -Eq '^...$' with timestamped runtime log format
  7. Traced the new doctor-three-tools-product-path-before-tui mutation — target cardinality guard, byte-change guard, dual named-red assertion (anchored gate + reason)
  8. Verified Dockerfile unchanged (no drift in base image, Bun pin, non-root, XDG_RUNTIME_DIR)
  9. Read report §Correction — honest root-cause disclosure of fix: fail closed when Grok CommHub MCP is unavailable #822 minified-source-stack loophole
  10. git log --oneline base..main -- <11 files> = 0 for each — no drift
  11. git merge-tree base main pr-825 | grep -cE '^<<<<<<< |^======= |^>>>>>>> ' = 0
  12. Secret sweep on delta files: 0 real credentials

No approve, no merge, no deploy.

@vansin

vansin commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Draft,所以我不合,只留 review。本机 bun test src/runtime/grok-build-cli-home.test.ts41 pass / 0 fail / 292 expect,合 main 无冲突。

这个 PR 要解决的问题,我认同它的描述

A bare command = "bun" makes recovery depend on whichever PATH happened to launch tmux, producing a false-healthy inbound-only node.

「能收不能发」是最难被发现的一种坏 —— 节点在 hub 上心跳正常、状态 idle、看起来完全健康,只有在它需要主动说话时才暴露。今晚我恰好在另一件事上碰到同一个形状(判一个静默节点是否存活),所以这条我读得很实。

resolveGrokCommhubMcpCommand 的做法也对:拒绝含 \0\r\n 的命令、逐个 realpathSync + isFile + accessSync(X_OK)返回规范化绝对路径、解析不到就抛。

🔴 一处我追了三跳才放下的怀疑,和一条它引出的真问题

怀疑(我错了,记录下来)

assertGrokCommhubMcpDoctor 要求一条字面检查标签:

const requiredChecks = ["command found", "server started", "handshake OK", "4 tools discovered"];

agent-network/src/node-server.ts 注册的是 6 个 tool:

commhub_reply  commhub_report_status  commhub_send_task
commhub_send_message  commhub_get_all_status  commhub_upload_file

看起来是 6 vs 4 —— 而后果很重:对不上就抛,节点起不来

追下去发现我错了。 grok 这条路写的是 ANET_COMMHUB_MODE = "outbound-only"(grok-build-cli-home.ts:1455),而 node-server 在这个模式下按 OUTBOUND_TOOL_NAMES 过滤(:255)。那个集合:

// agent-network/src/outbound-tool-names.ts
export const OUTBOUND_TOOL_NAMES = new Set([
  "commhub_send_task", "commhub_send_message",
  "commhub_get_all_status", "commhub_upload_file",
]);   // ← 正好 4 个

「4」是对的。 一个很像缺陷的东西,追到第三跳才化掉 —— 记在这里,免得下一个人重走一遍。

🔴 但由此引出的真问题:这个 4 是硬编码的,而同一个集合已经因为这件事错过一次

outbound-tool-names.ts 自己的文件头写着:

tests/test235-grok-mcp-outbound-only asserted a hard-coded copy of three names.
commhub_upload_file shipped in #693 and made it four, so that assertion has been wrong on main — and nothing reported it, because no workflow and neither qa.sh list runs test235.

也就是说:这个 tool 集合上一次增加成员时,一个硬编码的数字就已经悄悄失效过一次了。#825 把同样的硬编码放进了一个后果更重的位置 —— 上次错的是一条没人跑的测试断言,这次错的会是节点启动失败

下一个 outbound tool 落地的那一天,所有 grok 共存节点会同时起不来,而错误信息是「readiness failed: 4 tools discovered」—— 指向一个数字,不指向原因。

修法是现成的、而且这个常量正是为此存在的(文件头:「tests need to assert against it」):

import { OUTBOUND_TOOL_NAMES } from "…/outbound-tool-names";
const expectedTools = `${OUTBOUND_TOOL_NAMES.size} tools discovered`;
const requiredChecks = ["command found", "server started", "handshake OK", expectedTools];

⚠️ 跨包 import 的可行性要你确认(agent-node → agent-network 的源码 import 在别处有先例:test798 的镜像就为此 COPY 了 agent-node/src)。如果不便,退一步也行:4 提成一个具名常量并在注释里指向 outbound-tool-names.ts,至少让下一个人改的时候能搜到。

一条关于测试的观察

4 tools discovered 这条断言,目前只被 fake 验证过 —— fake-grok.mjs:218probe.ts:92 都是自己造 doctor 输出。fake 是照着断言写的,所以它必然对得上。 这不是这个 PR 的失误(真 grok 不可能进 CI),但值得写进注释:这一条的正确性依赖「真 grok 的 doctor 输出格式与 fake 一致」,而那一点没有任何自动化在看。

最后:这个 PR 是一条 5 层 stack 的唯一出口

#825  base=main            ← 只有它能到 main
 └ #826 → #830 → {#836, #867}

上面四个的分支分别领先 main 9 / 10 / 17 / 19 个提交。#825 不合,那 19 个提交一个都到不了 main。 五个全是 draft,我尊重这个标记不去合 —— 但这条拓扑值得知道:grok 共存那一整块的交付是单点。

@vansin

vansin commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

复核:这条 rebase 到今天的 main 是干净的,且不引入任何新的门失败

本地做的,没有动这个分支(force-push 一个别人也在用的 draft 分支不是我该自作主张的事)。
可复现:

git fetch origin fix/813-grok-mcp-readiness-v3
git branch -f probe/825 FETCH_HEAD && git switch probe/825
git rebase origin/main       # rc=0,0 冲突

基线 origin/main = 66029a55,本分支 9 个未落地补丁,落后 main 120 个提交。

拿今天 main 上的 21 道门判 rebase 后的树

过 17 · 红 4。但红的 4 条要先分环境,否则会把不属于这条 PR 的账算到它头上:

rebase 后的 #825 干净的 origin/main(对照) 判定
check-doc-symbol-pins.py . rc=1 rc=1 main 本来就红(见 #1001
check-docs-site-drift.py rc=1 rc=1 main 本来就红(站点没重新部署)
check-test-file-coverage.py rc=0 rc=0 🔴 我第一次量错了,见下
check-published-build-paths.py rc=2 rc=2 我漏传参数,CI 是带 <pkg>@<tag> 跑的

这条 PR 引入的新失败是 0。 剩下的都是 main 的账或我的量具问题。

🔴 我在这次测量里踩的两个坑,写下来免得下一个人重踩

① 把脚本拷到 /tmp 再跑,量的就不是这棵树。
check-test-file-coverage.py 第 40 行是 REPO = Path(__file__).resolve().parents[2]
/tmp/g2.py 跑必然 IndexError: 2它在原位是 rc=0、250 个测试文件…0 个漏网
一个「红」如果来自我的调用方式,它和真红长得一模一样。

printf "%s %s" "$(basename $g)" "$?" 里的 $? 不是那个命令的退出码。
词展开是从左到右的,$(basename …) 先跑完并把 $? 覆盖成它自己的 0,
于是我一度得到「三条全 rc=0」,与事实相反。和「管道后 $? 抓的是 tee」同一族。
正确写法:

python3 "$@" >/dev/null 2>&1; rc=$?; printf "%-40s rc=%s\n" "$name" "$rc"

对这个栈的意思

#825 → #826 → #830 → {#836, #867} 是一条链,只有 #825 的 base 是 main
上面四个的 base 都是各自父分支(合了不会进 main,见 #856 的分诊)。
所以栈只能自底向上落,而这条已经具备 rebase 条件:干净、无新增红。

⚠️ 它仍是 draft,且上次 CI 只跑了 10 个 check(今天一个 PR 跑 13–14 个)。
要真判它,得先 rebase 并 push、让它在今天这套门下重跑一遍 —— 那一步会改这个分支,
我不擅自做。

vansin added a commit that referenced this pull request Aug 18, 2026
)

发现方式:在本地把 #825 rebase 到 main、拿今天 21 道门判它,做对照组时
发现**干净的 origin/main(66029a55)自己就有两道红**,跟 #825 无关。
这条修其中一道。

    python3 scripts/check-doc-symbol-pins.py .        # qa.yml:309 的原文调用
    SYMBOL-PIN: RED(扫到 15 个 pin,判定 5 个,跳过 10 个,漂移 1 个)
      🔴 docs/architecture.md 锚文本点名 `dashboardReleaseTag`,钉在
         agent-network/bin/cli.ts#L1569,但那一行是: //
         `dashboardReleaseTag` 现在在第 1585 行(共 4 处)

🔴 **门抓到 1 条,人核同一句话抓到 3 条** —— 另外两条落在它「跳过的 10 个」里:

  ① URL `cli.ts#L1569`  → 真值 1585(门抓到的那条)
  ② 锚文本 `cli.ts:347` → 真值 1585
     **同一个引用,文本和 URL 写着两个不同的错数字。** 门只看 URL,
     而 `:347` 是渲染出来给人读的那半 —— 只修 URL 会留下用户实际看到的那个错。
  ③ `cli.ts:2386` 声称是 `sub === "dashboard"` 分支 → L2386 实际是 `stat.isFile()`,
     真正的分支在 **6279**(`} else if (sub === "dashboard" || sub === "dash") {`)

顺带把同句那条 grep 断言写精确。原文是「全 source grep `/dashboard` 0 hit」并链到
`server/src/server.ts` —— 实测 server.ts 里**有 1 处命中**,但它是第 287 行的注释
`any future HEAD/Range/dashboard-proxy`,不是路由;`server/src/index.ts` 才是 0 hit。
结论没变(确实没有 `/dashboard` 路由),但下一个人 grep 到那一处会以为这句话错了,
所以把「0 hit」改成写清楚那 1 处是什么。

改后:`SYMBOL-PIN: OK(扫到 15 个 pin,判定 5 个,跳过 10 个,漂移 0 个)` rc=0。
doc-claims / doc-source-pins / docs-integrity / doc-symbol-anchors /
no-memory-slugs / mcp-tool-anchors 全部 rc=0。

main 上另一道红(`check-docs-site-drift.py`,2 个页面没重新部署)不在这条里 ——
那要跑 `vercel --prod`,是对外动作,单独报。

Co-authored-by: t <internlmorg@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
vansin added a commit that referenced this pull request Aug 18, 2026
…1004)

* fix: fail closed on broken Grok CommHub MCP

* test: exercise Grok CommHub MCP readiness handshake

* test: add isolated Grok MCP readiness gate

* test: exercise the real Grok MCP doctor

* docs: report Grok vendor MCP readiness

* test: gate Grok readiness on product startup path

* test: keep Grok gate dependency layers source-independent

* test: anchor Grok readiness to runtime events

* docs: record exact Grok MCP readiness evidence

* fix(grok): 把 #825 的 MCP 就绪 fail-closed 落到 main(rebase + 补两处今天新门抓到的)

#825(`fix(grok): fail closed when CommHub MCP is not ready`)从 2026-08-15 起是
draft,落后 main 120 个提交,上次 CI 只跑了 10 个 check(今天一个 PR 跑 13–23 个)。
它是 grok 栈 `#825#826#830 → {#836,#867}` 的**唯一出口**,它不落地,
上面四条谁都到不了 main。

**没有 force-push 那条共享 draft 分支** —— 本 PR 从我自己的分支发,内容是它那 9 个
补丁 rebase 到 `origin/main`(`rc=0`,0 冲突)再加下面两处修补。

## 今天新合的两道门抓到了它两处

上一轮我测过「#825 引入新失败 0」,那句话当时是真的 —— 但那两道门是**今天晚些时候**
才合进 main 的,门槛抬高之后它就红了:

  1. `test-suite-registration`(#1003)
     `tests/test813-grok-mcp-readiness/` 是新增套件,没有任何 CI 会跑它。
     🔴 这条特别值得修而不是豁免:**#825 的目的就是「MCP 没就绪时 fail closed」,
     而验证这件事的套件如果不进 CI,fail-closed 这个保证就没有任何东西持续守着。**
     照 test831 的同构做法接进 qa.yml:两处 `paths` + build/run 两个 step。
     ⚠️ 它的 Dockerfile 收的 build-arg 叫 `SOURCE_COMMIT`(不是 831 的
     `TEST831_SOURCE_COMMIT`),容器里由 `ENV TEST813_SOURCE_COMMIT` 承接 ——
     照抄 831 的参数名会静默拿不到值。

  2. `doc-symbol-pins`(#1002)
     `docs/message-lifecycle.md` 把 `shouldSkipMessage` 钉在 `cli.ts#L4639`,
     而 #825 给 cli.ts 加了 54 行,真值现在是 **4662**。
     文本 `cli.ts:4639` 和 URL `#L4639` 是同一句话的两副面孔,两处都改 ——
     只改 URL 会留下渲染出来给人读的那个错数字。

## 验证

    rebase 到 origin/main            rc=0,0 冲突,9 个补丁
    test-suite-registration          rc=0   suites=198 registered=34 orphans=164 new=0
    doc-symbol-pins                  rc=0
    l1-paths-sync / qa-trigger-coverage / workflow-structure / docs-integrity /
    no-escaped-comments              全部 rc=0

`registered` 从 33 变 34、`orphans` 仍是 164 —— 新套件是接进 CI 了,不是塞进基线蒙混。

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(test813): 一条从写下起就是空的变异 —— 接进 CI 的第一次运行就把它照出来了

把 test813 接进 CI(本 PR 上一个提交)之后,它在 GitHub runner 上第一次运行就红:

    MCP_READINESS_PASS tools=commhub_get_all_status,commhub_send_message,commhub_send_task,commhub_upload_file
    FAIL: mutation survived: upload-tool-removed
    MCP_READINESS_PASS tools=…(同一份列表,commhub_upload_file 仍在)

不是产品没拦住,是**变异根本没发生**:

    sed -i '/^[[:space:]]*"commhub_upload_file",[[:space:]]*$/d' agent-network/src/node-server.ts

那个模式要求整行只有 `"commhub_upload_file",`,而真实那行是
`      name: "commhub_upload_file",`(对象字面量的字段)。实测命中 **0** ——
在 origin/main 上也是 0,**它从写下那天起就没匹配过任何一行**。
套件此前不在任何 CI 里(#861 说的 164 个孤儿之一),所以没人见过它红。

删整行会破坏对象字面量语法,改成改名:

    sed -i 's/name: "commhub_upload_file"/name: "commhub_upload_file_MUT"/' …

干跑验证:文件确实变了(`243:      name: "commhub_upload_file_MUT",`),已还原。

## 顺带堵住这一类,不只这一条

给 `expect_red` 加空变异防护:跑完变异命令后,若两个源文件**都**没被改动就直接判红,
并说清楚是「sed 模式和源码对不上」而不是「产品没拦住」。

这两种结论指向完全不同的下一步 —— **去改产品 vs 去改测试** —— 而它们在没有这道防护
时打印出来是同一句 `mutation survived`。

同一个套件里另外三条 sed 都逐条查过,各自命中 1(其中打 fake-grok.mjs 那条还自带
`grep -Fxc … -eq 1` 基数守卫),只有这一条是空的。

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: t <internlmorg@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@vansin

vansin commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

这条的内容已经落到 main 了(#100489a6164e),因此关闭

不是被别人的 PR 顺带带走的,是我把它 rebase 到 origin/main 之后单独落的 ——
没有 force-push 这条分支,本分支的历史保持原样。

🔴 先更正我在 #1004 正文里写错的一句

我在那条 PR 里写过「合并之后 git cherry 会把它判成 patch-equivalent」。实测不是:

git cherry origin/main fix/813-grok-mcp-readiness-v3
→ 9 个仍未落地 / 0 个已等价

原因是 #1004squash 合并,落在 main 上的是一个提交,它的 patch-id 和这条分支
那 9 个各自的 patch-id 都对不上。git cherry 抗 squash 的前提是「同一批改动被原样
squash」,而我在落地时还改了两处(见下),所以它更对不上。

⇒ 判「落地了没有」不能只看 git cherry,这里按内容验。

按内容验(这才是证据)

本分支相对其 merge-base 动了 11 个文件,11 个在 main 上全部存在
再逐行看「本分支新增的行,在 main 上找不找得到」:

文件 新增行 在 main 上找不到
agent-node/src/cli.ts 23 0
agent-node/src/runtime/grok-build-cli-home.ts 64 0
agent-node/src/runtime/grok-build-cli-home.test.ts 56 0
tests/test813-grok-mcp-readiness/run.sh 129 1

那唯一 1 行是我故意换掉的

sed -i '/^[[:space:]]*"commhub_upload_file",[[:space:]]*$/d' agent-network/src/node-server.ts

落地时改的两处,都是今天新合的门抓到的

  1. tests/test813-grok-mcp-readiness/ 接进了 CI(qa.yml 里 7 处引用)。
    理由:这条 PR 的目的是「MCP 没就绪时 fail closed」,而验证它的套件如果不在任何 CI 里,
    这个保证就没有任何东西持续守着。它此前是 194 个测试套件里只有 21 个被 CI 引用;其中 106 个孤儿近 30 天仍在维护 —— 需要一次分类 #861 那 164 个孤儿套件之一。

  2. 那条 upload-tool-removed 变异从写下起就是空的。 模式要求整行只有
    "commhub_upload_file",,真实那行是 name: "commhub_upload_file",(对象字面量
    的字段),在这条分支上和在 origin/main 上命中都是 0
    套件不在 CI 里,所以三天没人见过它红 —— 接进 CI 的第一次运行就红了

    FAIL: mutation survived: upload-tool-removed
    

    改成改名(删整行会破坏对象字面量语法),并给 expect_red 加了空变异防护:
    变异跑完若两个源文件都没变就直接判红,并说清是「sed 模式和源码对不上」而不是
    「产品没拦住」—— 这两种结论指向完全不同的下一步。

对这个栈的后续

#825 → #826 → #830 → {#836, #867}#825 这一层落地后,#826 现在可以直接 rebase 到
origin/main
,不再需要等它下面那层。栈的其余部分我会照同样的方式逐层往上落:
本地 rebase → 跑今天这套门 → 从我自己的分支发 PR → 不动原分支。

@vansin vansin closed this Aug 18, 2026
vansin added a commit that referenced this pull request Aug 18, 2026
* feat(grok): add strict repo-read copresence profile

* docs(tests): record restacked Grok repo-read evidence

* feat(grok): 把 #826 的 strict repo-read 共存档位落到 main

grok 栈 `#825#826#830 → {#836,#867}` 的第二层。#825 已由 #1004 落地
(main `89a6164e`),所以这一层现在可以直接对 main 落。

**没有 force-push #826 那条共享 draft 分支**,本 PR 从我自己的分支发。

## rebase 方式:只取它自己那两个提交

直接 `git rebase origin/main` **会冲突** —— 冲在 `436b4bdb fix: fail closed on broken
Grok CommHub MCP`,那是 **#825 自己的提交**,已经以 squash 形式在 main 上了,而 #826
的历史里还带着它。

正确做法是把它自己那部分接到 main 上:

    git rebase --onto origin/main <#825-head> <#826-head>     rc=0

它自己的两个提交:

    4496835 feat(grok): add strict repo-read copresence profile
    1149676 docs(tests): record restacked Grok repo-read evidence

结果 12 files changed, 284 insertions(+), 36 deletions(-)。

## 今天这套门抓到一处

`doc-symbol-pins`:`docs/message-lifecycle.md` 把 `shouldSkipMessage` 钉在
`cli.ts#L4662`,而本层又往 `cli.ts` 加了行,真值变成 **4669**。文本和 URL 两处都改。

🔴 **这是同一条 pin 今晚第二次漂**(#1004 刚把它从 4639 改到 4662)。
往 `cli.ts` 这种热点文件钉**行号**,几乎每个碰它的 PR 都会把它顶漂 ——
改成只钉符号、不钉行号才是根治,但那是另一条改动,不在这里顺手做。

其余全绿:test-suite-registration(suites=198 registered=34 orphans=164 new=0)/
doc-source-pins / no-escaped-comments / no-memory-slugs / home-path-baseline /
public-script-safety / l1-paths-sync / copresence-profile-pin。

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: t <internlmorg@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[grok-copresence] recovery can leave inbound Hub healthy while TUI CommHub MCP is dead

1 participant