-
Notifications
You must be signed in to change notification settings - Fork 9
test(ci): pin #767 Bun preflight behavior #770
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| test766 — Bun package-runner preflight behavior gate | ||
| Date: 2026-08-13 Asia/Shanghai | ||
|
|
||
| Verdict: PASS | ||
|
|
||
| Coordinates | ||
| ----------- | ||
| BASE_COMMIT=3b049160b2db542d06a32988a530f0b21f773c96 | ||
| SOURCE_COMMIT=885a74d7c5418d0ee20de46c728396a367ec41cf | ||
| IMAGE_ID=sha256:60994c0f0a088167e7f5ed77f74fc5fefbde57be414bd9a23b1764f48dc746bc | ||
| FOCUSED_RUN_LOG_SHA256=1779a95c74afdd17670d2311dc99f09605b671ff1ecfc8dac9a209c4b0de55f8 | ||
| L1_RUN_LOG_SHA256=fca1054ab0b7604a0d86e2a1f212e7d78b7214c2702c53e0e4d4a62d7b310a03 | ||
|
|
||
| Scope | ||
| ----- | ||
| - Test/CI only: no product, package, dist, deployment, database, or runtime | ||
| configuration changes. | ||
| - Adds test766 to the active scripts/qa.sh L1 list. | ||
| - Pins the behavior selected by #767; it does not decide whether #766 should | ||
| later adopt the separately tested automatic `bun x` fallback design. | ||
|
|
||
| Focused Docker result | ||
| --------------------- | ||
| The image used Bun 1.3.14 pinned by digest: | ||
| oven/bun:1.3.14@sha256:e10577f0db68676a7024391c6e5cb4b879ebd17188ab750cf10024a6d700e5c4 | ||
|
|
||
| Observed output: | ||
| source_commit=885a74d7c5418d0ee20de46c728396a367ec41cf | ||
| PASS bun-only, neither, and bunx behavior matrix | ||
| PASS agent-network production build | ||
| PASS mutation old-or-allows-bun-only witnessed red at its named behavior | ||
| PASS mutation valid-bunx-blocked witnessed red at its named behavior | ||
| RESULT pass=4 fail=0 | ||
|
|
||
| Behavior matrix | ||
| --------------- | ||
| 1. `bun` without `bunx`: nonzero before spawn, precise bun-only diagnosis and | ||
| manual symlink remediation, with no healthy-start banner. | ||
| 2. Neither executable: nonzero with the generic Bun installation guidance, | ||
| without the bun-only diagnosis or healthy-start banner. | ||
| 3. `bunx` available: the production CLI crosses the preflight, passes exact | ||
| argv `--bun @sleep2agi/commhub-server@0.9.0-preview.29`, and reaches the | ||
| healthy-Hub banner against the fixture server. | ||
|
|
||
| Witnessed-red mutations | ||
| ----------------------- | ||
| - `old-or-allows-bun-only`: restores the former permissive | ||
| `!bunx && !bun` guard. The clean bun-only probe is green; the mutation reaches | ||
| spawn and fails the named preflight behavior. | ||
| - `valid-bunx-blocked`: forces the guard to reject a valid bunx installation. | ||
| The clean bunx probe is green; the mutation prevents runner capture and the | ||
| healthy banner, so the named behavior turns red. | ||
| - Each mutation first requires an exact single production target and confirms | ||
| the changed text. A missing or duplicated target fails before classification. | ||
|
|
||
| Active L1 regression | ||
| -------------------- | ||
| Command: | ||
| bash scripts/qa.sh --l1 | ||
|
|
||
| Result on SOURCE_COMMIT: | ||
| 15/15 active L1 suites passed | ||
| test766: RESULT pass=4 fail=0 | ||
| ALL PASS in 107s | ||
| measured wall time: 107.24s | ||
|
|
||
| Provenance | ||
| ---------- | ||
| - The focused image recorded TEST766_SOURCE_COMMIT equal to SOURCE_COMMIT. | ||
| - The six files copied into the focused image under | ||
| tests/test766-bunx-preflight were compared byte-for-byte with SOURCE_COMMIT: | ||
| PROVENANCE total=6 fail=0. | ||
| - scripts/qa.sh is exercised by the host L1 runner and is not copied into the | ||
| focused image; its inclusion of test766 is proven by the 15-suite output. | ||
|
|
||
| Honest limits | ||
| ------------- | ||
| - The healthy path uses an HTTP fixture after the real CLI has selected and | ||
| invoked bunx. It proves preflight and argv wiring, not registry download or | ||
| the full commhub-server implementation; existing Hub E2E suites remain the | ||
| authority for those behaviors. | ||
| - This report does not claim a production rollout, npm publication, or a final | ||
| product decision for bun-only installations. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| FROM oven/bun:1.3.14@sha256:e10577f0db68676a7024391c6e5cb4b879ebd17188ab750cf10024a6d700e5c4 | ||
|
|
||
| WORKDIR /repo | ||
| ARG TEST766_SOURCE_COMMIT=unknown | ||
| ENV TEST766_SOURCE_COMMIT=$TEST766_SOURCE_COMMIT | ||
|
|
||
| COPY agent-network/package.json /repo/agent-network/package.json | ||
| RUN cd /repo/agent-network && bun install --ignore-scripts | ||
|
|
||
| COPY agent-network /repo/agent-network | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a checkout contains the ignored AGENTS.md reference: AGENTS.md:L7-L7 Useful? React with 👍 / 👎. |
||
| COPY tests/lib /repo/tests/lib | ||
| COPY tests/test766-bunx-preflight /repo/tests/test766-bunx-preflight | ||
| RUN chmod +x /repo/tests/test766-bunx-preflight/run.sh \ | ||
| /repo/tests/test766-bunx-preflight/fake-bun-only/bun \ | ||
| /repo/tests/test766-bunx-preflight/fake-bunx/bunx | ||
|
|
||
| CMD ["bash", "/repo/tests/test766-bunx-preflight/run.sh"] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # test766 — Bun package-runner preflight | ||
|
|
||
| Commits the three behavioral cases chosen by #767: | ||
|
|
||
| 1. `bun` exists and `bunx` does not: fail before spawn with the precise | ||
| remediation message; | ||
| 2. neither exists: retain the generic Bun installation message; | ||
| 3. `bunx` exists: cross the preflight, preserve the exact package argv, and | ||
| reach the CLI's healthy-Hub banner. | ||
|
|
||
| Two witnessed-red mutations restore the old permissive OR and make the guard | ||
| reject a valid bunx installation. This suite is listed in `scripts/qa.sh` L1; | ||
| it is not an uncalled local probe. | ||
|
|
||
| The healthy-path HTTP fixture covers CLI wiring and does not claim to replace | ||
| the repository's real commhub-server E2E suites. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| #!/bin/sh | ||
| exit 0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| #!/bin/sh | ||
| set -eu | ||
|
|
||
| capture=${TEST766_CAPTURE:?TEST766_CAPTURE is required} | ||
| printf '%s\n' "$@" > "$capture" | ||
| exec /usr/local/bin/bun /repo/tests/test766-bunx-preflight/fixture-server.ts "$@" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| const port = Number(process.env.PORT || "27668"); | ||
| const capture = process.env.TEST766_CAPTURE; | ||
| if (!capture) throw new Error("TEST766_CAPTURE is required"); | ||
|
|
||
| const server = Bun.serve({ | ||
| hostname: "127.0.0.1", | ||
| port, | ||
| fetch(request) { | ||
| const path = new URL(request.url).pathname; | ||
| if (path === "/health") { | ||
| return Response.json({ ok: true, version: "test766-bunx" }); | ||
| } | ||
| if (path === "/api/auth/login") { | ||
| return Response.json({ ok: false, error: "invalid credentials" }); | ||
| } | ||
| if (path === "/api/auth/register") { | ||
| return Response.json({ | ||
| ok: true, | ||
| token: "utok_test766_fixture", | ||
| user: { user_id: "user-766", username: "admin" }, | ||
| }); | ||
| } | ||
| return Response.json({ ok: false, error: "not found" }, { status: 404 }); | ||
| }, | ||
| }); | ||
|
|
||
| await Bun.write(`${capture}.pid`, String(process.pid)); | ||
|
|
||
| function stop() { | ||
| server.stop(true); | ||
| process.exit(0); | ||
| } | ||
| process.on("SIGTERM", stop); | ||
| process.on("SIGINT", stop); | ||
| await new Promise(() => {}); |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,142 @@ | ||
| #!/usr/bin/env bash | ||
| set -Eeuo pipefail | ||
|
|
||
| ROOT=/repo | ||
| ART=/artifacts | ||
| PASS=0 | ||
| FAIL=0 | ||
| mkdir -p "$ART" | ||
| source "$ROOT/tests/lib/safe-rm.sh" | ||
|
|
||
| ok(){ PASS=$((PASS+1)); printf 'PASS %s\n' "$*"; } | ||
| bad(){ FAIL=$((FAIL+1)); printf 'FAIL %s\n' "$*"; } | ||
|
|
||
| probe_bun_only(){ | ||
| local case_root home log rc | ||
| case_root=$(mktemp -d /tmp/test766-bun-only.XXXXXX) | ||
| home="$case_root/home" | ||
| log="$case_root/cli.log" | ||
| mkdir -p "$home" | ||
| set +e | ||
| HOME="$home" PATH="$ROOT/tests/test766-bunx-preflight/fake-bun-only:/usr/bin:/bin" \ | ||
| /usr/local/bin/bun "$ROOT/agent-network/bin/cli.ts" hub start \ | ||
| --host 127.0.0.1 --port 27668 --username admin --password StrongPassw0rd \ | ||
| >"$log" 2>&1 | ||
| rc=$? | ||
| set -e | ||
| if [[ "$rc" -eq 0 ]] \ | ||
| || ! grep -Fq '找到了 bun,但没有 bunx' "$log" \ | ||
| || ! grep -Fq 'ln -s "$(command -v bun)"' "$log" \ | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The bun-only probe matches only the source half of the suggested Useful? React with 👍 / 👎. |
||
| || grep -Fq 'Starting CommHub Server' "$log"; then | ||
| cat "$log" >&2 | ||
| safe_rm_rf "$case_root" | ||
| return 1 | ||
| fi | ||
| safe_rm_rf "$case_root" | ||
| } | ||
|
|
||
| probe_neither(){ | ||
| local case_root home log rc | ||
| case_root=$(mktemp -d /tmp/test766-neither.XXXXXX) | ||
| home="$case_root/home" | ||
| log="$case_root/cli.log" | ||
| mkdir -p "$home" | ||
| set +e | ||
| HOME="$home" PATH="/usr/bin:/bin" \ | ||
| /usr/local/bin/bun "$ROOT/agent-network/bin/cli.ts" hub start \ | ||
| --host 127.0.0.1 --port 27668 --username admin --password StrongPassw0rd \ | ||
| >"$log" 2>&1 | ||
| rc=$? | ||
| set -e | ||
| if [[ "$rc" -eq 0 ]] \ | ||
| || ! grep -Fq 'anet hub start requires the Bun runtime' "$log" \ | ||
| || grep -Fq '找到了 bun,但没有 bunx' "$log" \ | ||
| || grep -Fq 'Starting CommHub Server' "$log"; then | ||
| cat "$log" >&2 | ||
| safe_rm_rf "$case_root" | ||
| return 1 | ||
| fi | ||
| safe_rm_rf "$case_root" | ||
| } | ||
|
|
||
| probe_bunx(){ | ||
| local case_root home capture log cli_pid fixture_pid rc | ||
| case_root=$(mktemp -d /tmp/test766-bunx.XXXXXX) | ||
| home="$case_root/home" | ||
| capture="$case_root/runner.argv" | ||
| log="$case_root/cli.log" | ||
| mkdir -p "$home" | ||
|
|
||
| HOME="$home" TEST766_CAPTURE="$capture" \ | ||
| PATH="$ROOT/tests/test766-bunx-preflight/fake-bunx:/usr/bin:/bin" \ | ||
| /usr/local/bin/bun "$ROOT/agent-network/bin/cli.ts" hub start \ | ||
| --host 127.0.0.1 --port 27668 --username admin --password StrongPassw0rd \ | ||
| >"$log" 2>&1 & | ||
| cli_pid=$! | ||
|
|
||
| for _ in $(seq 1 100); do | ||
| [[ -s "$capture" ]] \ | ||
| && grep -Fq 'Server running on http://127.0.0.1:27668' "$log" \ | ||
| && break | ||
| sleep 0.05 | ||
| done | ||
|
|
||
| rc=0 | ||
| [[ -s "$capture" ]] || rc=1 | ||
| [[ "$(sed -n '1p' "$capture" 2>/dev/null || true)" == "--bun" ]] || rc=1 | ||
| grep -Fxq '@sleep2agi/commhub-server@0.9.0-preview.29' "$capture" || rc=1 | ||
|
Comment on lines
+86
to
+87
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The healthy-path probe only requires Useful? React with 👍 / 👎. |
||
| grep -Fq 'Server running on http://127.0.0.1:27668' "$log" || rc=1 | ||
|
|
||
| kill "$cli_pid" 2>/dev/null || true | ||
| fixture_pid=$(cat "$capture.pid" 2>/dev/null || true) | ||
| [[ -n "$fixture_pid" ]] && kill "$fixture_pid" 2>/dev/null || true | ||
| wait "$cli_pid" 2>/dev/null || true | ||
| if [[ "$rc" -ne 0 ]]; then | ||
| cat "$log" >&2 || true | ||
| sed -n '1,20p' "$capture" >&2 || true | ||
| safe_rm_rf "$case_root" | ||
| return 1 | ||
| fi | ||
| safe_rm_rf "$case_root" | ||
| } | ||
|
|
||
| probe_all(){ | ||
| probe_bun_only | ||
| probe_neither | ||
| probe_bunx | ||
| } | ||
|
|
||
| expect_red(){ | ||
| local name="$1"; shift | ||
| if "$@" >"$ART/$name.log" 2>&1; then | ||
| bad "mutation $name survived" | ||
| else | ||
| tail -50 "$ART/$name.log" | ||
| ok "mutation $name witnessed red at its named behavior" | ||
| fi | ||
| } | ||
|
|
||
| printf 'source_commit=%s\n' "${TEST766_SOURCE_COMMIT:-unknown}" | ||
| cd "$ROOT" | ||
|
|
||
| probe_all | ||
| ok "bun-only, neither, and bunx behavior matrix" | ||
| (cd agent-network && bun run build) | ||
| ok "agent-network production build" | ||
|
|
||
| cp agent-network/bin/cli.ts /tmp/test766-cli.orig | ||
|
|
||
| [[ "$(grep -Fc 'if (!commandExists("bunx")) {' agent-network/bin/cli.ts)" -eq 1 ]] | ||
| sed -i 's/if (!commandExists("bunx")) {/if (!commandExists("bunx") \&\& !commandExists("bun")) {/' agent-network/bin/cli.ts | ||
| grep -Fq 'if (!commandExists("bunx") && !commandExists("bun")) {' agent-network/bin/cli.ts | ||
| expect_red old-or-allows-bun-only probe_bun_only | ||
| cp /tmp/test766-cli.orig agent-network/bin/cli.ts | ||
|
|
||
| [[ "$(grep -Fc 'if (!commandExists("bunx")) {' agent-network/bin/cli.ts)" -eq 1 ]] | ||
| sed -i 's/if (!commandExists("bunx")) {/if (true || !commandExists("bunx")) {/' agent-network/bin/cli.ts | ||
| grep -Fq 'if (true || !commandExists("bunx")) {' agent-network/bin/cli.ts | ||
| expect_red valid-bunx-blocked probe_bunx | ||
| cp /tmp/test766-cli.orig agent-network/bin/cli.ts | ||
|
|
||
| printf 'RESULT pass=%s fail=%s\n' "$PASS" "$FAIL" | ||
| [[ "$FAIL" -eq 0 ]] | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because only
package.jsonis present whenbun installruns, the image resolves every ranged dependency from the registry instead of using the committedagent-network/package-lock.json; copying the repository afterward is too late. A new compatible dependency release can therefore make this active L1 suite pass or fail differently for the same source commit, undermining the recorded reproducibility guarantees. Copy the lockfile before installation and use a frozen-lockfile install.AGENTS.md reference: AGENTS.md:L25-L25
Useful? React with 👍 / 👎.