From 8b25565f4ec6a616e9982c8d2670df22bce8081d Mon Sep 17 00:00:00 2001 From: filipeforattini Date: Wed, 30 Sep 2026 01:19:40 -0300 Subject: [PATCH] chore: remover o harness de Vercel Sandbox do fork MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Este fork não tem credenciais Vercel nem intenção de usar o Sandbox, mas o AGENTS.md declarava `pnpm test:sandbox` como o gate antes de entregar qualquer mudança. O efeito prático era mandar todo agente para um gate inexecutável e deixar o gate real — as duas lanes locais — como um "fallback". Remove os onze arquivos do harness, as duas entradas de script do package.json e as referências em AGENTS.md, README.md, no README do test262 e nos dois planos sob .red/code/plans. O gate declarado passa a ser as lanes plain e sanitizada locais. O harness é código mantido em upstream/main, então esta remoção é uma divergência deliberada do fork e vai conflitar nas próximas sincronizações; a resolução é descartar o harness de novo. SCRIPTC_PORTABLE_ONLY continua nos testes que o consultam, mas nada o define mais: era o script de Sandbox que o ligava. Os contratos nativos de host passam a rodar sempre localmente. --- .../2026-09-26-backend-libcall-parity.md | 2 +- .../2026-09-30-estado-e-proximos-passos.md | 2 +- AGENTS.md | 30 +- Dockerfile.sandbox | 65 -- README.md | 28 +- package.json | 2 - scripts/sandbox-bootstrap.sh | 63 - scripts/sandbox-command.mjs | 39 - scripts/sandbox-config.mjs | 240 ---- scripts/sandbox-image.mjs | 100 -- scripts/sandbox-platform.mjs | 32 - scripts/sandbox-test.mjs | 1035 ----------------- tests/harness/runtime-tree-shaking.test.ts | 11 +- tests/harness/sandbox-command.test.ts | 78 -- tests/harness/sandbox-config.test.ts | 269 ----- tests/harness/sandbox-dockerfile.test.ts | 23 - tests/harness/sandbox-platform.test.ts | 52 - tests/test262/README.md | 2 +- 18 files changed, 19 insertions(+), 2054 deletions(-) delete mode 100644 Dockerfile.sandbox delete mode 100644 scripts/sandbox-bootstrap.sh delete mode 100644 scripts/sandbox-command.mjs delete mode 100644 scripts/sandbox-config.mjs delete mode 100644 scripts/sandbox-image.mjs delete mode 100644 scripts/sandbox-platform.mjs delete mode 100644 scripts/sandbox-test.mjs delete mode 100644 tests/harness/sandbox-command.test.ts delete mode 100644 tests/harness/sandbox-config.test.ts delete mode 100644 tests/harness/sandbox-dockerfile.test.ts delete mode 100644 tests/harness/sandbox-platform.test.ts diff --git a/.red/code/plans/2026-09-26-backend-libcall-parity.md b/.red/code/plans/2026-09-26-backend-libcall-parity.md index a8f3fca923..a7cbf38c9e 100644 --- a/.red/code/plans/2026-09-26-backend-libcall-parity.md +++ b/.red/code/plans/2026-09-26-backend-libcall-parity.md @@ -82,7 +82,7 @@ Concluído quando parent e child distinguirem ausência de IPC de canal desconec 2. Rodar `pnpm manifest` quando as tabelas de decisão do compiler mudarem e então `pnpm node-compat`; inspecionar o ledger interno, o artefato público e o backlog por mudanças não relacionadas. 3. Adicionar ou completar corpus diferencial por família. Cobrir sucesso, shapes importantes de erro, callback único, ordem/lifecycle, stdout, stderr e exit code. IPC precisa de fixtures parent/worker e crypto aleatório precisa testar invariantes, não bytes específicos. 4. Rodar testes focados por família sob `pnpm limit`, depois `cargo test` e `cargo clippy -- -D warnings` no runtime Rust, `pnpm node-compat:check`, build do workspace e gate de docs se artefatos públicos mudarem. -5. Rodar `pnpm test:sandbox`. Se credenciais Sandbox não existirem, rodar as lanes locais plain e sanitizada prescritas em `AGENTS.md`. Registrar separadamente qualquer limitação real do host para LLVM 22; não classificar ausência de toolchain como regressão de código. +5. Rodar as lanes locais plain e sanitizada prescritas em `AGENTS.md`, que são o gate completo deste fork. Registrar separadamente qualquer limitação real do host para LLVM 22; não classificar ausência de toolchain como regressão de código. Concluído quando o gerador não tiver órfãs, as 52 chamadas tiverem lowering Rust, o diff da implementação permanecer fora das lanes C/LLVM, os artefatos gerados estiverem limpos, o runtime Rust continuar sem `unsafe`, todos os gates disponíveis estiverem verdes e `git status` estiver limpo. diff --git a/.red/code/plans/2026-09-30-estado-e-proximos-passos.md b/.red/code/plans/2026-09-30-estado-e-proximos-passos.md index d564be7287..8180366ce9 100644 --- a/.red/code/plans/2026-09-30-estado-e-proximos-passos.md +++ b/.red/code/plans/2026-09-30-estado-e-proximos-passos.md @@ -35,7 +35,7 @@ Aberto: a Onda 6 do plano de libcall parity nunca rodou. Os critérios de regene 1. Zerar os 23 erros de clippy. Enquanto o gate declarado do runtime está vermelho, nenhum outro sinal é confiável. 2. Executar a Onda 6: corpus diferencial das 52 libcalls, uma família por commit. IPC precisa de fixtures parent e worker; aleatoriedade de crypto testa invariantes e não bytes específicos; toda callback precisa de prova de disparo único. O objetivo declarado é encontrar bugs nas implementações, não apenas produzir verde. -3. Rodar `pnpm test:sandbox`, que agora existe e conclui em cerca de quatro minutos. Ele substitui as lanes locais de várias horas que sofriam com OOM e contenção nesta máquina. +3. Rodar o gate completo, que são as duas lanes locais plain e sanitizada. O harness de Vercel Sandbox foi removido do fork em 30/09 por não haver credenciais nem intenção de usá-lo; as lanes locais são o único gate completo. Elas são longas nesta máquina, então isole `TMPDIR` e `SCRIPTC_CACHE_DIR` e não as rode sob `pnpm limit`. 4. Retomar o dogfooding nos alvos restantes, `red-dev` e `red-skills`, agora que redcode compila. ## Pegadinhas operacionais desta máquina diff --git a/AGENTS.md b/AGENTS.md index bf40f47bb2..7008527122 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,36 +9,14 @@ Write prose paragraphs and list items as single source lines and let editors and ## Build and test ```bash -pnpm install && pnpm -r build # build the workspace -pnpm test:sandbox # full gate: ~4m custom image, ~9m cold managed fallback +pnpm install && pnpm -r build # build the workspace +SCRIPTC_TEST_WORKERS=4 pnpm test # full gate: plain lane +SCRIPTC_TEST_WORKERS=4 SCRIPTC_SAN=1 pnpm test # full gate: sanitized lane ``` The ordinary workspace build does not rebuild packaged native artifacts. When changing native assembly/object emission or runtime-pack selection, install CMake, Ninja, the pinned LLVM 22 development package, and Zig 0.16.0, then run the matching `@scriptc/llvm-` and `@scriptc/runtime-` `build:native` scripts explicitly. The macOS full test suite also needs its generated artifacts. -Use focused local tests while iterating, then use `pnpm test:sandbox` whenever a -full validation gate is required. It loads Sandbox configuration from the -shell and `.env.local`, runs portable coverage across disposable Linux -Sandboxes, and retains the Darwin-native contracts on macOS. Linux hosts run -their supported native-clang contracts locally; other hosts retain those -checks in the Sandboxes. Both lanes green is the bar before shipping any -change. - -`VERCEL_OIDC_TOKEN` is the preferred credential. `VERCEL_TOKEN` remains -supported with explicit `VERCEL_TEAM_ID` and `VERCEL_PROJECT_ID`. The custom -`SCRIPTC_SANDBOX_IMAGE` is optional: without it, the gate starts from -`vercel/sandbox/universal` and installs the repository-pinned Node, pnpm, and -LLVM toolchain plus workspace dependencies before building. Team/project -selection never comes from the image reference. The legacy VCR image command -uses `VERCEL_TOKEN` or the existing Vercel CLI login for authentication; OIDC -claims can still provide its team/project scope. - -Only when Vercel Sandbox credentials are unavailable, run the slower local -fallback: - -```bash -SCRIPTC_TEST_WORKERS=4 pnpm test # plain lane -SCRIPTC_TEST_WORKERS=4 SCRIPTC_SAN=1 pnpm test # sanitized lane -``` +Use focused local tests while iterating, then run both full lanes locally before shipping. Both lanes green is the bar before shipping any change. Linux hosts run their supported native-clang contracts locally, and macOS retains the Darwin-native contracts. `SCRIPTC_TEST_WORKERS` caps the vitest worker pool so concurrent agents don't contend for cores. Direct local runs default to two workers, two nested native diff --git a/Dockerfile.sandbox b/Dockerfile.sandbox deleted file mode 100644 index 6192f52d47..0000000000 --- a/Dockerfile.sandbox +++ /dev/null @@ -1,65 +0,0 @@ -ARG NODE_VERSION=24.15.0 -FROM ubuntu:24.04 - -ARG NODE_VERSION -ARG PNPM_VERSION=11.1.3 -ARG ZIG_VERSION=0.16.0 - -ENV DEBIAN_FRONTEND=noninteractive - -RUN apt-get update \ - && apt-get install --yes --no-install-recommends \ - ca-certificates curl gnupg \ - && curl --fail --silent --show-error --location https://apt.llvm.org/llvm-snapshot.gpg.key \ - | gpg --dearmor --output /usr/share/keyrings/llvm-snapshot.gpg \ - && echo "deb [signed-by=/usr/share/keyrings/llvm-snapshot.gpg] https://apt.llvm.org/noble/ llvm-toolchain-noble-22 main" \ - > /etc/apt/sources.list.d/llvm22.list \ - && apt-get update \ - && apt-get install --yes --no-install-recommends \ - build-essential \ - ca-certificates \ - ccache \ - clang-22 \ - cmake \ - curl \ - git \ - libclang-rt-22-dev \ - libzstd-dev \ - llvm-22-dev \ - ninja-build \ - xz-utils \ - zlib1g-dev \ - && ln -sf clang-22 /usr/bin/clang \ - && ln -sf clang++-22 /usr/bin/clang++ \ - && test "$(llvm-config-22 --version)" = 22.1.8 \ - && rm -rf /var/lib/apt/lists/* - -RUN curl --fail --silent --show-error --location \ - --output /tmp/node.tar.xz \ - "https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.xz" \ - && curl --fail --silent --show-error --location \ - --output /tmp/SHASUMS256.txt \ - "https://nodejs.org/dist/v${NODE_VERSION}/SHASUMS256.txt" \ - && grep " node-v${NODE_VERSION}-linux-x64.tar.xz$" /tmp/SHASUMS256.txt \ - | sed 's#node-v[^ ]*-linux-x64.tar.xz#/tmp/node.tar.xz#' \ - | sha256sum --check --strict - \ - && tar -xJf /tmp/node.tar.xz --directory /usr/local --strip-components=1 \ - && rm /tmp/node.tar.xz /tmp/SHASUMS256.txt - -COPY scripts/install-zig.sh /tmp/install-zig.sh -RUN sh /tmp/install-zig.sh "$ZIG_VERSION" \ - && rm /tmp/install-zig.sh - -RUN npm install --global "pnpm@${PNPM_VERSION}" - -WORKDIR /workspace - -COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./ -COPY packages/cli/package.json packages/cli/package.json -COPY packages/cli/scripts/warm-cache.mjs packages/cli/scripts/warm-cache.mjs -COPY packages/cli/scripts/runtime-pack-host.mjs packages/cli/scripts/runtime-pack-host.mjs -COPY packages/compiler/package.json packages/compiler/package.json -COPY packages/runtime/package.json packages/runtime/package.json - -RUN pnpm install --frozen-lockfile \ - && chmod -R a+rwX /workspace diff --git a/README.md b/README.md index 5ba2cfef13..b1581226e0 100644 --- a/README.md +++ b/README.md @@ -202,8 +202,7 @@ not a claim established by the default selection. ```console $ pnpm install && pnpm -r build -$ vercel link && vercel env pull # writes a project-scoped VERCEL_OIDC_TOKEN -$ pnpm test:sandbox +$ SCRIPTC_TEST_WORKERS=4 pnpm test ``` The normal workspace build needs no local LLVM installation. To rebuild a @@ -212,22 +211,9 @@ development package on that target host, then run the matching `@scriptc/llvm-` and `@scriptc/runtime-` `build:native` scripts. The macOS full test suite also uses those generated artifacts. -`pnpm test:sandbox` loads `.env.local`, preflights Vercel authentication and -project access, and uses the managed `vercel/sandbox/universal` image by -default. It installs the repository-pinned Node, pnpm, and LLVM toolchain plus -ScriptC dependencies in each disposable Sandbox before building the uploaded -worktree. Set -`SCRIPTC_SANDBOX_IMAGE` to a fully qualified VCR reference only to use the -optional prebuilt image from `pnpm test:sandbox:image`. -The prebuilt image keeps the roughly four-minute fast path; cold managed-image -runs take longer because they install the pinned toolchain in each Sandbox. - -`VERCEL_OIDC_TOKEN` is preferred. For access-token authentication, set -`VERCEL_TOKEN`, `VERCEL_TEAM_ID`, and `VERCEL_PROJECT_ID`; team and project are -never inferred from `SCRIPTC_SANDBOX_IMAGE`. The legacy VCR command used by -`pnpm test:sandbox:image` cannot authenticate with an OIDC JWT, so image builds -use `VERCEL_TOKEN` when available or the existing Vercel CLI login; OIDC claims -still select the VCR team and project. The test corpus runs each program -under Node and as a compiled native binary, then compares stdout, stderr, and -exit codes byte for byte. The full gate also runs the corpus with -AddressSanitizer and the runtime reference-count audit. +The full gate is the two local lanes. `SCRIPTC_TEST_WORKERS` caps the vitest +worker pool so concurrent runs do not contend for cores. The test corpus runs +each program under Node and as a compiled native binary, then compares stdout, +stderr, and exit codes byte for byte. The sanitized lane +(`SCRIPTC_SAN=1 pnpm test`) runs the same corpus with AddressSanitizer and the +runtime reference-count audit. diff --git a/package.json b/package.json index 938c6eb4cb..0ca406cf60 100644 --- a/package.json +++ b/package.json @@ -21,8 +21,6 @@ "inventory:node": "node scripts/node-surface-inventory.mjs", "inventory:node:check": "node scripts/node-surface-inventory.mjs --check --offline", "test:cache-identity": "node tests/harness/cache-identity.mjs", - "test:sandbox": "node scripts/sandbox-test.mjs", - "test:sandbox:image": "node scripts/sandbox-image.mjs", "test:watch": "vitest", "limit": "node scripts/run-resource-limited.mjs", "lint": "node scripts/check-rust-file-lines.mjs && node scripts/gen-island-bootstrap.mjs --check && tsx scripts/gen-backend-libcalls.mjs --check && eslint packages/compiler/src packages/cli/src", diff --git a/scripts/sandbox-bootstrap.sh b/scripts/sandbox-bootstrap.sh deleted file mode 100644 index 54c06c8b15..0000000000 --- a/scripts/sandbox-bootstrap.sh +++ /dev/null @@ -1,63 +0,0 @@ -#!/bin/sh -set -eu - -node_version=$(tr -d '\r\n' < .node-version) -pnpm_version=$(sed -n 's/^ARG PNPM_VERSION=//p' Dockerfile.sandbox) -zig_version=$(sed -n 's/^ARG ZIG_VERSION=//p' Dockerfile.sandbox) -if [ -z "$node_version" ] || [ -z "$pnpm_version" ] || [ -z "$zig_version" ]; then - echo "could not read the pinned Node, pnpm, or Zig version" >&2 - exit 1 -fi - -# The managed image intentionally floats its distro and language runtimes. -# Install the same LLVM and exact Node/pnpm tuple as the custom image so -# differential oracles do not change when the managed image is refreshed. -. /etc/os-release -llvm_distro=${VERSION_CODENAME:?the Sandbox image must identify its Ubuntu release} -curl --fail --silent --show-error --location \ - https://apt.llvm.org/llvm-snapshot.gpg.key \ - --output /tmp/llvm-snapshot.gpg.key -gpg --dearmor --yes --output /tmp/llvm-snapshot.gpg /tmp/llvm-snapshot.gpg.key -sudo install -D -m 0644 /tmp/llvm-snapshot.gpg /etc/apt/keyrings/llvm-snapshot.gpg -echo "deb [signed-by=/etc/apt/keyrings/llvm-snapshot.gpg] https://apt.llvm.org/${llvm_distro}/ llvm-toolchain-${llvm_distro}-22 main" \ - | sudo tee /etc/apt/sources.list.d/llvm22.list >/dev/null - -sudo apt-get update --quiet=2 -sudo env DEBIAN_FRONTEND=noninteractive apt-get install --quiet=2 --yes --no-install-recommends \ - build-essential \ - ca-certificates \ - ccache \ - clang-22 \ - cmake \ - libclang-rt-22-dev \ - libzstd-dev \ - llvm-22-dev \ - ninja-build \ - xz-utils \ - zlib1g-dev -sudo rm -f /usr/local/bin/clang /usr/local/bin/clang++ -sudo ln -sf clang-22 /usr/bin/clang -sudo ln -sf clang++-22 /usr/bin/clang++ -test "$(llvm-config-22 --version)" = 22.1.8 - -node_archive="node-v${node_version}-linux-x64.tar.xz" -curl --fail --silent --show-error --location \ - "https://nodejs.org/dist/v${node_version}/${node_archive}" \ - --output "/tmp/${node_archive}" -curl --fail --silent --show-error --location \ - "https://nodejs.org/dist/v${node_version}/SHASUMS256.txt" \ - --output /tmp/node-SHASUMS256.txt -grep " ${node_archive}$" /tmp/node-SHASUMS256.txt \ - | sed "s# ${node_archive}# /tmp/${node_archive}#" \ - | sha256sum --check --strict - -sudo tar -xJf "/tmp/${node_archive}" --directory /usr/local --strip-components=1 -sh scripts/install-zig.sh "$zig_version" -sudo npm install --global "pnpm@${pnpm_version}" - -rm -f "/tmp/${node_archive}" /tmp/node-SHASUMS256.txt /tmp/llvm-snapshot.gpg.key /tmp/llvm-snapshot.gpg -sudo rm -rf /var/lib/apt/lists/* - -node --version -pnpm --version -zig version -clang --version | sed -n '1p' diff --git a/scripts/sandbox-command.mjs b/scripts/sandbox-command.mjs deleted file mode 100644 index 5efaa907c6..0000000000 --- a/scripts/sandbox-command.mjs +++ /dev/null @@ -1,39 +0,0 @@ -export const MAX_INLINE_SANDBOX_COMMAND_BYTES = 768; -// Outside the shell exit-code range, so an actual exit 125 stays a failure. -export const REMOTE_COMMAND_PENDING = 256; - -export const shellQuote = (value) => `'${value.replaceAll("'", `'"'"'`)}'`; - -/** Keep long shell programs out of the local CLI's argument vector. The - * same script records the remote exit status for either transport. */ -export function sandboxCommand(command, args, exitMarker) { - const statusPath = `/tmp/${exitMarker}.status`; - const script = - `${[command, ...args].map(shellQuote).join(" ")}; scriptc_status=$?; ` + - `printf '%s\\n' "$scriptc_status" > ${shellQuote(statusPath)}; ` + - `printf '\\n${exitMarker}%s\\n' "$scriptc_status"`; - const scriptPath = `/tmp/${exitMarker}.sh`; - const file = Buffer.byteLength(script, "utf8") > MAX_INLINE_SANDBOX_COMMAND_BYTES; - return { - script, - scriptPath, - statusPath, - file, - argv: file ? ["sh", scriptPath] : ["sh", "-c", script], - }; -} - -/** Allow a disconnected command to finish before probing its recorded exit. */ -export function sandboxStatusCommand(statusPath, exitMarker, waitSeconds = 20) { - if (!Number.isSafeInteger(waitSeconds) || waitSeconds < 0) { - throw new Error("status wait must be a nonnegative integer"); - } - return ( - `scriptc_wait=${waitSeconds}; ` + - `while test ! -s ${shellQuote(statusPath)} && test "$scriptc_wait" -gt 0; do ` + - `sleep 1; scriptc_wait=$((scriptc_wait - 1)); done; ` + - `scriptc_status=${REMOTE_COMMAND_PENDING}; test ! -s ${shellQuote(statusPath)} || ` + - `scriptc_status=$(cat ${shellQuote(statusPath)}); ` + - `printf '\\n${exitMarker}%s\\n' "$scriptc_status"` - ); -} diff --git a/scripts/sandbox-config.mjs b/scripts/sandbox-config.mjs deleted file mode 100644 index 4f3bd7cf0e..0000000000 --- a/scripts/sandbox-config.mjs +++ /dev/null @@ -1,240 +0,0 @@ -import { loadEnvFile } from "node:process"; -import { fileURLToPath } from "node:url"; - -const customImageExample = "vcr.vercel.com///:"; -export const defaultSandboxImage = "vercel/sandbox/universal"; -const localEnv = fileURLToPath(new URL("../.env.local", import.meta.url)); -let loadedLocalEnv = false; - -function loadLocalEnv() { - if (loadedLocalEnv) return; - loadedLocalEnv = true; - try { - // Node preserves variables already present in the process environment, - // allowing an agent or shell to override values from this local file. - loadEnvFile(localEnv); - } catch (error) { - if (error?.code !== "ENOENT") throw error; - } -} - -export function sandboxRunnerConfig(env) { - loadLocalEnv(); - const source = env ?? process.env; - return { - vcpus: source.SCRIPTC_SANDBOX_VCPUS ?? "8", - testWorkers: source.SCRIPTC_TEST_WORKERS ?? "2", - localTestWorkers: source.SCRIPTC_LOCAL_TEST_WORKERS ?? "2", - localCaseShards: source.SCRIPTC_LOCAL_CASE_SHARDS ?? "2", - sandboxTimeout: source.SCRIPTC_SANDBOX_TIMEOUT ?? "45m", - }; -} - -/** Select Sandbox credentials and scope without coupling either to the image. - * OIDC is project-scoped and takes precedence over legacy access tokens and - * their explicit environment scope. */ -export function sandboxVercelConfig(env) { - loadLocalEnv(); - const source = env ?? process.env; - const oidcToken = source.VERCEL_OIDC_TOKEN?.trim(); - const configuredTeam = source.VERCEL_TEAM_ID?.trim(); - const configuredProject = source.VERCEL_PROJECT_ID?.trim(); - if (!oidcToken && Boolean(configuredTeam) !== Boolean(configuredProject)) { - throw new Error( - "VERCEL_TEAM_ID and VERCEL_PROJECT_ID must be set together; " + - "Sandbox scope is independent of SCRIPTC_SANDBOX_IMAGE", - ); - } - const team = oidcToken ? undefined : configuredTeam; - const project = oidcToken ? undefined : configuredProject; - - const accessToken = source.VERCEL_TOKEN?.trim() || source.VERCEL_AUTH_TOKEN?.trim(); - const accessTokenSource = source.VERCEL_TOKEN?.trim() - ? "VERCEL_TOKEN" - : source.VERCEL_AUTH_TOKEN?.trim() - ? "VERCEL_AUTH_TOKEN" - : undefined; - const authSource = oidcToken - ? "VERCEL_OIDC_TOKEN" - : accessToken - ? accessTokenSource - : "Vercel CLI login"; - if (accessToken && !oidcToken && (!team || !project)) { - throw new Error( - `${authSource} requires VERCEL_TEAM_ID and VERCEL_PROJECT_ID for Sandbox access`, - ); - } - - return { - accessToken, - accessTokenSource, - authSource, - authToken: oidcToken || accessToken, - oidc: Boolean(oidcToken), - project, - scopeArgs: team && project ? ["--scope", team, "--project", project] : [], - scopeSource: oidcToken - ? "VERCEL_OIDC_TOKEN claims" - : team && project - ? "VERCEL_TEAM_ID + VERCEL_PROJECT_ID" - : "linked/default Vercel CLI project", - team, - }; -} - -/** Build a child environment that makes the selected authentication source - * unambiguous. The Vercel wrapper otherwise prefers a stored/access token to - * VERCEL_OIDC_TOKEN when both are available. */ -export function sandboxVercelEnvironment(config, env) { - const childEnv = { ...(env ?? process.env) }; - if (config.authToken) childEnv.VERCEL_AUTH_TOKEN = config.authToken; - if (config.oidc) delete childEnv.VERCEL_TOKEN; - return childEnv; -} - -/** Adapt the selected Sandbox credential to the legacy VCR command surface. - * Unlike the Sandbox CLI, `vercel vcr` rejects JWTs in VERCEL_TOKEN and cannot - * infer registry scope from OIDC claims. OIDC therefore supplies only VCR's - * scope; authentication uses an explicit access token or the Vercel CLI login. */ -export function sandboxVcrConfig(config, env) { - const childEnv = { ...(env ?? process.env) }; - delete childEnv.VERCEL_AUTH_TOKEN; - let authSource = "Vercel CLI login"; - const vcrAccessToken = /^\w+$/.test(config.accessToken ?? "") - ? config.accessToken - : undefined; - if (config.accessToken && !vcrAccessToken && !config.oidc) { - throw new Error( - `${config.accessTokenSource} is not a VCR-compatible access token; ` + - "use VERCEL_TOKEN with a classic access token or run `vercel login`", - ); - } - if (vcrAccessToken) { - childEnv.VERCEL_TOKEN = vcrAccessToken; - authSource = config.accessTokenSource; - } else { - delete childEnv.VERCEL_TOKEN; - } - - if (config.team && config.project) { - return { - authSource, - env: childEnv, - scopeArgs: ["--scope", config.team, "--project", config.project], - scopeSource: "VERCEL_TEAM_ID + VERCEL_PROJECT_ID", - }; - } - if (!config.oidc) { - return { - authSource, - env: childEnv, - scopeArgs: [], - scopeSource: "linked/default Vercel CLI project", - }; - } - - let claims; - try { - const segments = config.authToken?.split(".") ?? []; - if (segments.length !== 3 || !segments[1]) throw new Error("token is not a JWT"); - claims = JSON.parse(Buffer.from(segments[1], "base64url").toString("utf8")); - } catch (cause) { - throw new Error( - "VERCEL_OIDC_TOKEN must be a JWT with owner_id and project_id claims for VCR access", - { cause }, - ); - } - const team = typeof claims.owner_id === "string" ? claims.owner_id.trim() : ""; - const project = typeof claims.project_id === "string" ? claims.project_id.trim() : ""; - if (!team || !project) { - throw new Error( - "VERCEL_OIDC_TOKEN must contain owner_id and project_id claims for VCR access", - ); - } - return { - authSource, - env: childEnv, - scopeArgs: ["--scope", team, "--project", project], - scopeSource: "VERCEL_OIDC_TOKEN claims", - }; -} - -export function sandboxBootstrapCommand(customImage) { - if (customImage) return undefined; - return { - install: { - args: ["scripts/sandbox-bootstrap.sh"], - command: "sh", - workdir: "/workspace", - }, - prepareWorkspace: { - args: [ - "-c", - "sudo mkdir -p /workspace && sudo chown \"$(id -u):$(id -g)\" /workspace", - ], - command: "sh", - workdir: "/vercel", - }, - }; -} - -/** Reserve one Vitest worker for each concurrently scheduled side process - * without ever starving the case-sharded corpus or exceeding the configured - * per-Sandbox worker budget. Extra side processes share the reserved slots. */ -export function sandboxTestWorkerAllocation(workerCount, sideTaskCount) { - if (!Number.isInteger(workerCount) || workerCount < 1) { - throw new Error("workerCount must be a positive integer"); - } - if (!Number.isInteger(sideTaskCount) || sideTaskCount < 0) { - throw new Error("sideTaskCount must be a non-negative integer"); - } - const sideConcurrency = Math.min(sideTaskCount, Math.max(0, workerCount - 1)); - return { - caseWorkers: workerCount - sideConcurrency, - sideConcurrency, - }; -} - -export function sandboxImageConfig(env) { - loadLocalEnv(); - const source = env ?? process.env; - const configuredReference = source.SCRIPTC_SANDBOX_IMAGE?.trim(); - if (!configuredReference) { - return { - custom: false, - reference: defaultSandboxImage, - sandboxImage: defaultSandboxImage, - }; - } - - const match = /^vcr\.vercel\.com\/([^/]+)\/([^/]+)\/([^/:]+):([^/:]+)$/.exec( - configuredReference, - ); - if (!match) { - throw new Error( - `SCRIPTC_SANDBOX_IMAGE must be a fully qualified VCR image (${customImageExample})`, - ); - } - - const [, , , repository, tag] = match; - return { - custom: true, - reference: configuredReference, - repository, - tag, - // Preserve the full reference: Sandbox scope comes from Vercel auth, - // never from path segments in this image name. - sandboxImage: configuredReference, - }; -} - -export function requiredSandboxImageConfig(env) { - const config = sandboxImageConfig(env); - const reference = (env ?? process.env).SCRIPTC_SANDBOX_IMAGE?.trim(); - if (!reference) { - throw new Error( - `SCRIPTC_SANDBOX_IMAGE is required to build an image (for example: ${customImageExample})`, - ); - } - return config; -} diff --git a/scripts/sandbox-image.mjs b/scripts/sandbox-image.mjs deleted file mode 100644 index 356d2bbbbb..0000000000 --- a/scripts/sandbox-image.mjs +++ /dev/null @@ -1,100 +0,0 @@ -#!/usr/bin/env node -import { readFile } from "node:fs/promises"; -import { fileURLToPath } from "node:url"; -import { spawn } from "node:child_process"; -import { - requiredSandboxImageConfig, - sandboxVcrConfig, - sandboxVercelConfig, -} from "./sandbox-config.mjs"; -import { linuxAmd64ManifestDigest } from "./oci-manifest.mjs"; - -const root = fileURLToPath(new URL("../", import.meta.url)); -const { reference: image, repository, tag } = requiredSandboxImageConfig(); -const vercelConfig = sandboxVercelConfig(); -const vcrConfig = sandboxVcrConfig(vercelConfig); -const scopeArgs = vcrConfig.scopeArgs; -const nodeVersion = (await readFile(new URL("../.node-version", import.meta.url), "utf8")).trim(); - -function run(command, args, { capture = false } = {}) { - return new Promise((resolve, reject) => { - const child = spawn(command, args, { - cwd: root, - env: { ...vcrConfig.env, NO_UPDATE_NOTIFIER: "1" }, - stdio: capture ? ["ignore", "pipe", "pipe"] : "inherit", - }); - let stdout = ""; - let stderr = ""; - if (capture) { - child.stdout.setEncoding("utf8"); - child.stderr.setEncoding("utf8"); - child.stdout.on("data", (chunk) => (stdout += chunk)); - child.stderr.on("data", (chunk) => (stderr += chunk)); - } - child.on("error", reject); - child.on("exit", (code, signal) => { - if (code === 0) resolve(stdout); - else reject(new Error(`${command} exited ${signal ?? code}${stderr ? `\n${stderr.trim()}` : ""}`)); - }); - }); -} - -console.log( - `Authenticating Docker with ${vcrConfig.authSource} ` + - `(scope: ${vcrConfig.scopeSource})...`, -); -await run("vercel", ["vcr", "login", "docker", ...scopeArgs]); - -console.log(`Building and pushing ${image} for linux/amd64...`); -await run("docker", [ - "buildx", - "build", - "--platform", - "linux/amd64", - "--build-arg", - `NODE_VERSION=${nodeVersion}`, - "--file", - "Dockerfile.sandbox", - "--output", - `type=image,name=${image},push=true,oci-mediatypes=true,compression=zstd,compression-level=3,force-compression=true`, - ".", -]); - -const inspection = JSON.parse( - await run( - "docker", - ["buildx", "imagetools", "inspect", image, "--format", "{{json .}}"], - { capture: true }, - ), -); -let amd64Digest; -try { - amd64Digest = linuxAmd64ManifestDigest(inspection); -} catch (error) { - throw new Error(`could not find the linux/amd64 manifest for ${image}`, { - cause: error, - }); -} - -console.log("Waiting for VCR to prepare the image for Sandbox..."); -const deadline = Date.now() + 5 * 60_000; -while (Date.now() < deadline) { - const listing = JSON.parse( - await run( - "vercel", - ["vcr", "image", "ls", repository, "--format", "json", ...scopeArgs], - { capture: true }, - ), - ); - const manifest = listing.images?.find((candidate) => candidate.manifestDigest === amd64Digest); - if (manifest?.status === "ready") { - console.log(`${repository}:${tag} is ready for Vercel Sandbox.`); - process.exit(0); - } - if (manifest?.status === "unoptimized") { - throw new Error(`${repository}:${tag} is unoptimized; Vercel Sandbox requires linux/amd64`); - } - await new Promise((resolve) => setTimeout(resolve, 2000)); -} - -throw new Error(`VCR did not prepare ${repository}:${tag} within 5 minutes`); diff --git a/scripts/sandbox-platform.mjs b/scripts/sandbox-platform.mjs deleted file mode 100644 index c3e4a34b66..0000000000 --- a/scripts/sandbox-platform.mjs +++ /dev/null @@ -1,32 +0,0 @@ -/** - * Choose where tests that exercise the native host toolchain run. - * - * Darwin and Linux have supported native clang paths, so those files should - * test the contributor's actual host. Other hosts keep the coverage in the - * Linux Sandboxes. Darwin alone adds the compact kqueue/Mach-O contracts. - */ -export function sandboxHostSchedule(platform, invariantFiles) { - const runsNative = platform === "darwin" || platform === "linux"; - return { - darwinContracts: platform === "darwin", - localArtifactContracts: runsNative, - localInvariantFiles: runsNative ? invariantFiles : [], - remoteArtifactContracts: !runsNative, - remoteInvariantFiles: runsNative ? [] : invariantFiles, - }; -} - -/** - * Pin lane and scheduler identity even when the parent shell, .env.local, or - * Sandbox image already defines these variables. Empty values restore the - * unsharded, platform-native defaults; intentional remote and local shards - * override them at their call sites. - */ -export function sandboxLaneEnv(lane) { - return { - CI: "1", - SCRIPTC_PORTABLE_ONLY: "", - SCRIPTC_SAN: lane === "san" ? "1" : "", - SCRIPTC_TEST_SHARD: "", - }; -} diff --git a/scripts/sandbox-test.mjs b/scripts/sandbox-test.mjs deleted file mode 100644 index a2daafda73..0000000000 --- a/scripts/sandbox-test.mjs +++ /dev/null @@ -1,1035 +0,0 @@ -#!/usr/bin/env node -import { spawn, spawnSync } from "node:child_process"; -import { randomBytes } from "node:crypto"; -import { mkdtemp, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { pipeline } from "node:stream/promises"; -import { parseArgs } from "node:util"; -import { fileURLToPath } from "node:url"; -import { - sandboxBootstrapCommand, - sandboxImageConfig, - sandboxRunnerConfig, - sandboxTestWorkerAllocation, - sandboxVercelConfig, - sandboxVercelEnvironment, -} from "./sandbox-config.mjs"; -import { - sandboxHostSchedule, - sandboxLaneEnv, -} from "./sandbox-platform.mjs"; -import { - filterExistingWorktreePaths, - workspaceResetCommand, -} from "./worktree-files.mjs"; -import { REMOTE_COMMAND_PENDING, sandboxCommand, sandboxStatusCommand, shellQuote } from "./sandbox-command.mjs"; - -const root = fileURLToPath(new URL("../", import.meta.url)); -const laneCaseShardedFiles = [ - "tests/harness/differential.test.ts", - "tests/harness/llvm-differential.test.ts", - "tests/harness/npm.test.ts", - "tests/harness/server.test.ts", - "tests/harness/test262.test.ts", -]; -// Coverage analysis is frontend-only: SCRIPTC_SAN cannot change its result. -// It still case-shards across the selected lane so every corpus entry is -// checked, but running the same sweep in the second lane adds no coverage. -// The Rust differential is sanitizer-invariant for the opposite reason: the -// Rust backend refuses --sanitize outright, and the suite skips itself -// under SCRIPTC_SAN=1. -const invariantCaseShardedFiles = [ - "tests/harness/coverage.test.ts", - "tests/harness/rust-differential.test.ts", -]; -// Native-cache invalidation cases mutate process-wide compiler inputs and -// deliberately disable the immutable-toolchain memo used by the differential -// corpus. Keep them in their own Vitest process, matching CI: co-scheduling -// this file with the corpus can consume one of the corpus workers with the -// strict (and intentionally expensive) production probe path. -// -// They are sanitizer-invariant and use their own shard variable so the -// independent cases can spread across the once lane's Sandboxes without -// colliding with corpus case selection. -const cacheCaseShardedFiles = ["packages/compiler/src/backend/native-toolchain.test.ts"]; -const caseShardedFiles = [ - ...laneCaseShardedFiles, - ...invariantCaseShardedFiles, - ...cacheCaseShardedFiles, -]; - -// These files neither consume SCRIPTC_SAN nor delegate to a helper that does. -// Run their full coverage once. Files absent from this allowlist remain in -// both lanes by default, so a new test never silently loses sanitizer coverage. -const invariantRemoteFiles = [ - "packages/cli/test/flush.test.ts", - "packages/cli/test/paths.test.ts", - "packages/compiler/src/library/int-infer.test.ts", - "packages/compiler/test/cjs-lexer.test.ts", - "packages/compiler/test/emit-c.test.ts", - "packages/compiler/test/ir.test.ts", - "packages/compiler/test/llvm-runtime-abi.test.ts", - "packages/compiler/test/ts7/bench.test.ts", - "packages/compiler/test/ts7/coverage.test.ts", - "packages/compiler/test/ts7/facade.test.ts", - "packages/compiler/test/ts7/order-parity.test.ts", - "packages/compiler/test/ts7/parity.test.ts", - "packages/compiler/test/ts7/program-adapter.test.ts", - "packages/compiler/test/ts7/resolver-parity.test.ts", - // These C runtime units compile with ASan + SCR_RC_AUDIT themselves. - "packages/runtime/test/array.test.ts", - "packages/runtime/test/bytes.test.ts", - "packages/runtime/test/closure.test.ts", - "packages/runtime/test/inspect.test.ts", - "packages/runtime/test/json.test.ts", - "packages/runtime/test/map.test.ts", - "packages/runtime/test/path.test.ts", - "packages/runtime/test/regex.test.ts", - "tests/harness/island-surface.test.ts", - "tests/harness/library-mode.test.ts", - "tests/harness/library-profile.test.ts", - "tests/harness/linux-differential.test.ts", - "tests/harness/oci-manifest.test.ts", - "tests/harness/sandbox-config.test.ts", - "tests/harness/sandbox-platform.test.ts", - "tests/harness/shard.test.ts", - "tests/harness/smoke.test.ts", - "tests/harness/surface-manifest.test.ts", - "tests/harness/windows-differential.test.ts", - "tests/harness/worktree-files.test.ts", -]; - -// Full native-oracle coverage stays on the host where the expected answer -// really is Darwin-, libc-, architecture-, or linker-specific. Each test is -// already explicitly sanitized where useful, so a second flavor is identical. -const hostInvariantFiles = [ - "packages/compiler/test/cc-driver.test.ts", - "packages/runtime/test/lib.test.ts", - "packages/runtime/test/number.test.ts", - "packages/runtime/test/runtime.test.ts", - "packages/runtime/test/string.test.ts", - "packages/runtime/test/tonumber.test.ts", - "packages/runtime/test/url.test.ts", -]; -const hostSchedule = sandboxHostSchedule(process.platform, hostInvariantFiles); -const nativeHostInvariantFiles = hostSchedule.localInvariantFiles; -const remoteWorkspaceReset = workspaceResetCommand("/workspace"); - -// The full portable behavior of these suites runs remotely. A compact -// host-native contract additionally pins the places Darwin can disagree: -// object/archive ABI, Mach-O size classes, linker diagnostics, ucontext, -// and the kqueue event-loop arms under both ordinary and Apple-ASan builds. -// Non-Darwin hosts retain the full portable remote suites without trying -// to execute these macOS-specific contracts locally. -const hostLaneContractFiles = [ - "tests/harness/ffi.test.ts", - "tests/harness/island.test.ts", - "tests/harness/library-multi.test.ts", - "tests/harness/differential.test.ts", - "tests/harness/server.test.ts", - "tests/harness/dgram.test.ts", - "tests/harness/event-loop.test.ts", -]; -const hostLaneContractPattern = [ - "calls the manifest-bound archive across every v1 ABI class", - "a missing FFI symbol is an SC5004 diagnostic", - "deep island recursion on a fiber is a catchable RangeError", - "M1: external definitions equal the declared set exactly", - "M2: independent state and collects", - "M6: four threads, one archive", - "M7: thread-instanced and runtime-localized archives compose", - "M8: M6 under ASan", - "net-echo", - "udp-loopback-pair", - "1564-fs-watch.ts", - "1470-child-lifecycle.ts", - "2963-child-fork-dispatch/main.ts", - "read-all: chunked writes with delays, then EOF", -].join("|"); -const hostInvariantContractFiles = [ - "tests/harness/island.test.ts", - "tests/harness/library-mode.test.ts", - "tests/harness/regex.test.ts", -]; -const hostInvariantContractPattern = [ - "static hello-world stays in its size class", - "K1/K2/K8: scalar round-trips, symbol exactness, ambient audit", - "K3: buffer round-trips \\+ lifetime, auto-reset posture", - "K5: a trap delivers to the sink exactly once, host survives", - "K10: K4 under ASan \\+ RC audit", - "K10: K5/K7 under ASan", - "regex-free programs never reference the regex runtime", -].join("|"); - -// Logically portable acceptance suites whose oracle lives in an external -// worktree that is intentionally not uploaded. Run them locally in both -// flavors, sharding suites whose individual cases are independently addressable. -const localLaneFiles = [ - "tests/harness/prettier-e2e.test.ts", - "tests/harness/portless-e2e.test.ts", -]; -const localCaseShardedFiles = ["tests/harness/vercel-e2e.test.ts"]; - -const { values } = parseArgs({ - options: { - help: { type: "boolean", short: "h" }, - keep: { type: "boolean" }, - lane: { type: "string", default: "both" }, - "remote-only": { type: "boolean" }, - shards: { type: "string", default: "8" }, - }, -}); - -if (values.help) { - console.log(`Run the scriptc test suite across Vercel Sandboxes. - -Usage: - pnpm test:sandbox [--lane plain|san|both] [--shards 8] [--remote-only] [--keep] - -Environment: - VERCEL_OIDC_TOKEN Preferred project-scoped Sandbox credential - VERCEL_TOKEN Access-token fallback; also set VERCEL_TEAM_ID + VERCEL_PROJECT_ID - SCRIPTC_SANDBOX_IMAGE Optional fully qualified VCR image (default: vercel/sandbox/universal) - SCRIPTC_SANDBOX_VCPUS vCPUs per sandbox (default: 8) - SCRIPTC_SANDBOX_TIMEOUT sandbox and command timeout (default: 45m) - SCRIPTC_TEST_WORKERS Vitest workers per sandbox (default: 4) - SCRIPTC_LOCAL_TEST_WORKERS Vitest workers per local lane (default: 2) - SCRIPTC_LOCAL_CASE_SHARDS local shards per external suite lane (default: 2)`); - process.exit(0); -} - -const imageConfig = sandboxImageConfig(); -const { sandboxImage: image } = imageConfig; -const bootstrapCommand = sandboxBootstrapCommand(imageConfig.custom); -const vercelConfig = sandboxVercelConfig(); -const vercelProcessEnv = sandboxVercelEnvironment(vercelConfig); -const { - vcpus, - testWorkers, - localTestWorkers, - localCaseShards, - sandboxTimeout, -} = sandboxRunnerConfig(); - -if (!["plain", "san", "both"].includes(values.lane)) { - throw new Error(`--lane must be plain, san, or both (got ${values.lane})`); -} -const shardCount = Number(values.shards); -if (!Number.isInteger(shardCount) || shardCount < 1 || shardCount > 10) { - throw new Error(`--shards must be an integer from 1 to 10 (got ${values.shards})`); -} - -const lanes = values.lane === "both" ? ["plain", "san"] : [values.lane]; -const remoteWorkerCount = Number(testWorkers); -if (!Number.isInteger(remoteWorkerCount) || remoteWorkerCount < 1) { - throw new Error(`SCRIPTC_TEST_WORKERS must be a positive integer (got ${testWorkers})`); -} -const fileWorkers = "1"; -const localCaseShardCount = Number(localCaseShards); -if (!Number.isInteger(localCaseShardCount) || localCaseShardCount < 1) { - throw new Error(`SCRIPTC_LOCAL_CASE_SHARDS must be a positive integer (got ${localCaseShards})`); -} -const onceLane = lanes.includes("plain") ? "plain" : lanes[0]; -const specialFiles = [ - ...caseShardedFiles, - ...invariantRemoteFiles, - ...hostInvariantFiles, - ...localLaneFiles, - ...localCaseShardedFiles, -]; -if (new Set(specialFiles).size !== specialFiles.length) { - throw new Error("a test file cannot belong to more than one execution path"); -} -const nonce = `${Date.now().toString(36)}-${randomBytes(3).toString("hex")}`; -const workers = lanes.flatMap((lane) => - Array.from({ length: shardCount }, (_, offset) => { - const shard = offset + 1; - return { - lane, - shard, - label: `${lane} ${shard}/${shardCount}`, - name: `scriptc-${lane}-${shard}-${nonce}`, - }; - }), -); -const created = new Set(); -const children = new Set(); -let cleanupPromise; -let handlingSignal = false; - -function lineWriter(destination, prefix, handleLine) { - let buffered = ""; - return { - write(chunk) { - buffered += chunk; - const lines = buffered.split(/\r?\n/); - buffered = lines.pop() ?? ""; - for (const line of lines) { - if (!handleLine?.(line)) destination.write(`${prefix}${line}\n`); - } - }, - end() { - if (buffered && !handleLine?.(buffered)) destination.write(`${prefix}${buffered}\n`); - }, - }; -} - -function run( - command, - args, - { - baseEnv = process.env, - env = {}, - exitMarker, - idleTimeoutMs, - label, - quiet = false, - timeoutMs, - } = {}, -) { - return new Promise((resolve, reject) => { - const child = spawn(command, args, { - cwd: root, - env: { ...baseEnv, NO_UPDATE_NOTIFIER: "1", ...env }, - stdio: quiet ? "ignore" : ["ignore", "pipe", "pipe"], - }); - children.add(child); - const prefix = label ? `[${label}] ` : ""; - let remoteExitCode; - let timedOut = false; - let timeoutReason = ""; - let killTimeout; - const stopForTimeout = (reason) => { - if (timedOut) return; - timedOut = true; - timeoutReason = reason; - child.kill("SIGTERM"); - // A network-stalled CLI may not honor SIGTERM promptly. Escalate so a - // wall timeout also bounds the time spent waiting for the close event. - killTimeout = setTimeout(() => child.kill("SIGKILL"), 5_000); - }; - const timeout = - timeoutMs === undefined - ? undefined - : setTimeout(() => { - stopForTimeout(`after ${Math.round(timeoutMs / 1000)}s`); - }, timeoutMs); - let idleTimeout; - const resetIdleTimeout = () => { - if (idleTimeoutMs === undefined) return; - if (idleTimeout !== undefined) clearTimeout(idleTimeout); - idleTimeout = setTimeout( - () => stopForTimeout(`after ${Math.round(idleTimeoutMs / 1000)}s without output`), - idleTimeoutMs, - ); - }; - resetIdleTimeout(); - const stdout = lineWriter(process.stdout, prefix, (line) => { - if (!exitMarker) return false; - const match = new RegExp(`^${exitMarker}(\\d+)$`).exec(line); - if (!match) return false; - remoteExitCode = Number(match[1]); - return true; - }); - const stderr = lineWriter(process.stderr, prefix); - if (!quiet) { - child.stdout.setEncoding("utf8"); - child.stderr.setEncoding("utf8"); - child.stdout.on("data", (chunk) => { - resetIdleTimeout(); - stdout.write(chunk); - }); - child.stderr.on("data", (chunk) => { - resetIdleTimeout(); - stderr.write(chunk); - }); - } - child.on("error", reject); - child.on("close", (code, signal) => { - if (timeout !== undefined) clearTimeout(timeout); - if (idleTimeout !== undefined) clearTimeout(idleTimeout); - if (killTimeout !== undefined) clearTimeout(killTimeout); - children.delete(child); - stdout.end(); - stderr.end(); - if (timedOut) { - reject( - Object.assign(new Error(`${label ?? command} timed out ${timeoutReason}`), { - code: "SCRIPTC_RUN_TIMEOUT", - }), - ); - } else if (code !== 0) { - reject(new Error(`${label ?? command} exited ${signal ?? code}`)); - } else if (exitMarker && remoteExitCode === undefined) { - reject(new Error(`${label ?? command} did not report its remote exit status`)); - } else if (remoteExitCode !== undefined && remoteExitCode !== 0) { - reject(Object.assign(new Error(`${label ?? command} remote command exited ${remoteExitCode}`), { remoteExitCode })); - } else { - resolve(); - } - }); - }); -} - -const scopeArgs = vercelConfig.scopeArgs; -const vercel = ([group, command, ...args], options) => - run("vercel", [group, command, ...scopeArgs, ...args], { - ...options, - baseEnv: vercelProcessEnv, - }); -// `vercel sandbox exec` does not propagate the remote process's exit code. -// Print a per-command nonce after it finishes and enforce that status here. -const execIn = async ( - worker, - command, - args, - env = {}, - task = "", - wallTimeoutMs = 15 * 60_000, - workdir = "/workspace", - idleTimeoutMs = 90_000, -) => { - const envArgs = Object.entries(env).flatMap(([key, value]) => ["--env", `${key}=${value}`]); - const exitMarker = `__SCRIPTC_REMOTE_EXIT_${randomBytes(12).toString("hex")}__`; - const prepared = sandboxCommand(command, args, exitMarker); - const { statusPath } = prepared; - const label = task ? `${worker.label} ${task}` : worker.label; - if (prepared.file) { - const localScript = join(temp, `${exitMarker}.sh`); - await writeFile(localScript, prepared.script, { mode: 0o600 }); - try { - await vercel(["sandbox", "copy", localScript, `${worker.name}:${prepared.scriptPath}`], { - idleTimeoutMs: 60_000, - label: `${label} command`, - timeoutMs: 2 * 60_000, - }); - } finally { - await rm(localScript, { force: true }); - } - } - const commandArgs = [ - "sandbox", - "exec", - "--timeout", - sandboxTimeout, - "--workdir", - workdir, - ...envArgs, - worker.name, - ...prepared.argv, - ]; - const deadline = Date.now() + wallTimeoutMs; - try { - await vercel(commandArgs, { - exitMarker, - idleTimeoutMs, - label, - timeoutMs: wallTimeoutMs, - }); - } catch (error) { - if (error.remoteExitCode !== undefined) throw error; - console.warn(`[${label}] CLI completion was not confirmed (${error.message}); checking the remote command status...`); - for (;;) { - const remaining = deadline - Date.now(); - if (remaining <= 0) throw new Error(`${label} did not confirm completion before its timeout`, { cause: error }); - const probeMarker = `__SCRIPTC_REMOTE_PROBE_${randomBytes(12).toString("hex")}__`; - const probeScript = sandboxStatusCommand(statusPath, probeMarker, Math.min(20, Math.floor(remaining / 1000))); - try { - await vercel( - ["sandbox", "exec", "--timeout", "1m", "--workdir", workdir, worker.name, "sh", "-c", probeScript], - { - exitMarker: probeMarker, - idleTimeoutMs: 30_000, - label: `${label} status`, - timeoutMs: Math.min(60_000, remaining), - }, - ); - return; - } catch (probeError) { - if (probeError.remoteExitCode !== REMOTE_COMMAND_PENDING) throw probeError; - console.log(`[${label}] remote command has not recorded completion; waiting...`); - } - } - } -}; - -async function preflight() { - const customImage = imageConfig.custom ? "custom VCR image" : "managed fallback image"; - console.log("Sandbox preflight:"); - console.log(` auth: ${vercelConfig.authSource}`); - console.log(` scope: ${vercelConfig.scopeSource}`); - console.log(` image: ${image} (${customImage})`); - console.log(` shape: ${workers.length} sandboxes, ${vcpus} vCPUs each`); - - try { - await run("vercel", ["--version"], { - baseEnv: vercelProcessEnv, - label: "preflight CLI", - timeoutMs: 30_000, - }); - } catch (cause) { - throw new Error( - "Sandbox preflight could not run the repository's Vercel CLI; run `pnpm install` first", - { cause }, - ); - } - - try { - await vercel(["sandbox", "list", "--limit", "1"], { - label: "preflight access", - timeoutMs: 60_000, - }); - } catch (cause) { - const credentialHint = vercelConfig.oidc - ? "Refresh VERCEL_OIDC_TOKEN with `vercel env pull` and verify that it belongs to a Sandbox-enabled project." - : vercelConfig.authToken - ? "Verify VERCEL_TOKEN, VERCEL_TEAM_ID, and VERCEL_PROJECT_ID and confirm that project can use Sandbox." - : "Run `vercel login` and `vercel link`, or provide VERCEL_OIDC_TOKEN."; - throw new Error( - `Sandbox preflight could not access the selected Vercel project. ${credentialHint}`, - { cause }, - ); - } - console.log("Sandbox preflight passed.\n"); -} - -async function createArchive(path) { - const git = spawn("git", ["ls-files", "--cached", "--others", "--exclude-standard", "-z"], { - cwd: root, - env: { ...process.env, COPYFILE_DISABLE: "1" }, - stdio: ["ignore", "pipe", "inherit"], - }); - const tarArgs = [ - ...(process.platform === "darwin" ? ["--no-xattrs", "--no-mac-metadata"] : []), - "--null", - "-T", - "-", - "-czf", - path, - ]; - const tar = spawn("tar", tarArgs, { - cwd: root, - env: { ...process.env, COPYFILE_DISABLE: "1" }, - stdio: ["pipe", "inherit", "inherit"], - }); - children.add(git); - children.add(tar); - const pack = pipeline(git.stdout, filterExistingWorktreePaths(root), tar.stdin); - const wait = (child, name) => - new Promise((resolve, reject) => { - child.on("error", reject); - child.on("exit", (code, signal) => { - children.delete(child); - if (code === 0) resolve(); - else reject(new Error(`${name} exited ${signal ?? code}`)); - }); - }); - await Promise.all([wait(git, "git ls-files"), wait(tar, "tar"), pack]); -} - -async function createWorker(worker) { - const args = [ - "sandbox", - "create", - "--name", - worker.name, - "--image", - image, - "--timeout", - sandboxTimeout, - "--vcpus", - vcpus, - "--non-persistent", - ]; - for (let attempt = 1; attempt <= 2; attempt++) { - try { - await vercel(args, { - idleTimeoutMs: 60_000, - label: worker.label, - timeoutMs: 2 * 60_000, - }); - return; - } catch { - console.warn(`[${worker.label}] create completion was not confirmed; checking the Sandbox...`); - try { - await execIn(worker, "true", [], {}, "create status", 60_000, "/"); - return; - } catch (error) { - if (attempt === 2) throw error; - console.warn(`[${worker.label}] Sandbox is not reachable; retrying creation once...`); - } - } - } -} - -async function uploadArchive(worker, archive) { - for (let attempt = 1; attempt <= 2; attempt++) { - try { - await vercel(["sandbox", "copy", archive, `${worker.name}:/tmp/worktree.tar.gz`], { - idleTimeoutMs: 60_000, - label: worker.label, - timeoutMs: 2 * 60_000, - }); - return; - } catch { - console.warn(`[${worker.label}] copy completion was not confirmed; checking the remote archive...`); - try { - // Listing every member also verifies the gzip stream reached its - // footer; a merely non-empty, partially uploaded file is rejected. - await execIn( - worker, - "sh", - ["-c", "tar -tzf /tmp/worktree.tar.gz >/dev/null"], - {}, - "copy status", - 60_000, - "/", - ); - return; - } catch (error) { - if (attempt === 2) throw error; - console.warn(`[${worker.label}] remote archive is absent or incomplete; retrying the copy once...`); - } - } - } -} - -async function allWorkers(phase, task, concurrency = workers.length) { - const started = Date.now(); - console.log(`\n${phase} (${workers.length} sandboxes)...`); - const results = []; - for (let offset = 0; offset < workers.length; offset += concurrency) { - results.push(...(await Promise.allSettled(workers.slice(offset, offset + concurrency).map(task)))); - } - const failures = results.filter((result) => result.status === "rejected"); - if (failures.length) { - for (const failure of failures) console.error(failure.reason); - throw new Error(`${phase} failed for ${failures.length} sandbox${failures.length === 1 ? "" : "es"}`); - } - console.log(`${phase} completed in ${((Date.now() - started) / 1000).toFixed(1)}s`); -} - -async function runTaskQueue(tasks, concurrency) { - for (let offset = 0; offset < tasks.length; offset += concurrency) { - await Promise.all(tasks.slice(offset, offset + concurrency).map((task) => task())); - } -} - -async function cleanup() { - if (values.keep || created.size === 0) return; - if (!cleanupPromise) { - cleanupPromise = (async () => { - console.log(`\nRemoving ${created.size} disposable sandbox${created.size === 1 ? "" : "es"}...`); - const results = await Promise.allSettled( - [...created].map((name) => - vercel(["sandbox", "remove", name], { - label: name, - quiet: true, - timeoutMs: 60_000, - }).then(() => created.delete(name)), - ), - ); - const failures = results.filter((result) => result.status === "rejected"); - if (failures.length) { - console.error(`Failed to remove ${failures.length} sandbox${failures.length === 1 ? "" : "es"}.`); - } - })(); - } - await cleanupPromise; -} - -for (const signal of ["SIGINT", "SIGTERM"]) { - process.once(signal, () => { - if (handlingSignal) return; - handlingSignal = true; - for (const child of children) child.kill("SIGTERM"); - if (!values.keep && created.size) { - console.log(`\nRemoving ${created.size} disposable sandbox${created.size === 1 ? "" : "es"}...`); - spawnSync( - "vercel", - ["sandbox", "remove", ...scopeArgs, ...created], - { - cwd: root, - env: { ...vercelProcessEnv, NO_UPDATE_NOTIFIER: "1" }, - stdio: "inherit", - timeout: 30_000, - }, - ); - } - process.exit(signal === "SIGINT" ? 130 : 143); - }); -} - -const temp = await mkdtemp(join(tmpdir(), "scriptc-sandbox-test-")); -const archive = join(temp, "worktree.tar.gz"); -const suiteStarted = Date.now(); -let failure; - -try { - await preflight(); - console.log( - `Running ${lanes.join("+")} corpus lanes in ${workers.length} ${vcpus}-vCPU sandboxes from ${image} (${shardCount} shards/lane).`, - ); - console.log("Packing the exact tracked + untracked, non-ignored worktree..."); - await createArchive(archive); - - const remote = (async () => { - await allWorkers("Creating", async (worker) => { - created.add(worker.name); - await createWorker(worker); - }); - - await allWorkers( - "Uploading worktree", - (worker) => uploadArchive(worker, archive), - 8, - ); - - await allWorkers("Preparing worktree", async (worker) => { - if (bootstrapCommand) { - await execIn( - worker, - bootstrapCommand.prepareWorkspace.command, - bootstrapCommand.prepareWorkspace.args, - {}, - "workspace", - 2 * 60_000, - bootstrapCommand.prepareWorkspace.workdir, - ); - } - await execIn( - worker, - remoteWorkspaceReset.command, - remoteWorkspaceReset.args, - {}, - "reset", - 2 * 60_000, - ); - await execIn( - worker, - "tar", - ["-xzf", "/tmp/worktree.tar.gz", "-C", "/workspace"], - {}, - "", - 2 * 60_000, - ); - if (bootstrapCommand) { - await execIn( - worker, - bootstrapCommand.install.command, - bootstrapCommand.install.args, - {}, - "bootstrap", - 15 * 60_000, - bootstrapCommand.install.workdir, - 5 * 60_000, - ); - } - await execIn(worker, "pnpm", ["install", "--frozen-lockfile"], {}, "", 2 * 60_000); - await execIn(worker, "pnpm", ["build"], {}, "", 2 * 60_000); - // Workspace builds deliberately do not rebuild packaged native artifacts. - // Every remote lane needs the Linux helper and runtime from this worktree. - await execIn(worker, "pnpm", ["--filter", "@scriptc/llvm-linux-x64-gnu", "build:native"], {}, "LLVM helper", 5 * 60_000); - await execIn( - worker, - "pnpm", - ["--filter", "@scriptc/runtime-linux-x64-gnu", "build:native"], - { CC: "zig", AR: "zig" }, - "runtime pack", - 5 * 60_000, - "/workspace", - 3 * 60_000, - ); - // Zig is a build-only dependency in this lane. Cross-target suites own - // the conditional Zig tests; exposing it here would silently expand the - // native-cache shard while that shard deliberately disables stable - // toolchain caching. - await execIn( - worker, - "sudo", - ["rm", "-f", "/usr/local/bin/zig"], - {}, - "runtime toolchain cleanup", - 60_000, - ); - }, imageConfig.custom ? workers.length : 8); - - await allWorkers("Testing", async (worker) => { - const sharedTestEnv = { - ...sandboxLaneEnv(worker.lane), - // Platform artifact contracts run against the native host below. - // Remote lanes retain every portable behavior assertion. - SCRIPTC_PORTABLE_ONLY: "1", - ...(worker.lane === "san" - ? { - // Match the macOS shipping lane: Apple ASan has no - // LeakSanitizer, while scriptc's RC audit owns leak - // detection (including its intentional-abandonment rules). - ASAN_OPTIONS: "detect_leaks=0", - } - : {}), - }; - const workerCaseFiles = [ - ...laneCaseShardedFiles, - ...(worker.lane === onceLane ? invariantCaseShardedFiles : []), - ]; - const runCacheCases = worker.lane === onceLane; - const { caseWorkers, sideConcurrency } = sandboxTestWorkerAllocation( - remoteWorkerCount, - runCacheCases ? 2 : 1, - ); - const cases = () => - execIn( - worker, - "pnpm", - ["test", "--reporter=dot", ...workerCaseFiles], - { - ...sharedTestEnv, - SCRIPTC_TEST_SHARD: `${worker.shard}/${shardCount}`, - SCRIPTC_TEST_WORKERS: String(caseWorkers), - }, - "cases", - ); - const files = () => - execIn( - worker, - "pnpm", - [ - "test", - "--reporter=dot", - "--passWithNoTests", - `--shard=${worker.shard}/${shardCount}`, - ...caseShardedFiles.map((file) => `--exclude=${file}`), - ...nativeHostInvariantFiles.map((file) => `--exclude=${file}`), - ...localLaneFiles.map((file) => `--exclude=${file}`), - ...localCaseShardedFiles.map((file) => `--exclude=${file}`), - ...(worker.lane === onceLane - ? [] - : [ - ...invariantRemoteFiles, - ...hostSchedule.remoteInvariantFiles, - ].map((file) => `--exclude=${file}`)), - ], - { - ...sharedTestEnv, - SCRIPTC_TEST_WORKERS: fileWorkers, - }, - "files", - ); - const cacheCases = () => - execIn( - worker, - "pnpm", - ["test", "--reporter=dot", ...cacheCaseShardedFiles], - { - ...sharedTestEnv, - SCRIPTC_CACHE_TEST_SHARD: `${worker.shard}/${shardCount}`, - // This is the strict production-path contract. Keep it out of - // the memoized corpus process and pin the opt-out explicitly, - // just as the GitHub Actions matrix does. - SCRIPTC_TEST_STABLE_TOOLCHAIN: "0", - SCRIPTC_TEST_WORKERS: "1", - }, - "cache", - ); - // The corpus owns the remaining pool while file/cache processes share - // the reserved side slots (serially when only one slot is available). - const sideTasks = [files, ...(runCacheCases ? [cacheCases] : [])]; - if (sideConcurrency === 0) { - await cases(); - await runTaskQueue(sideTasks, 1); - } else { - await Promise.all([ - cases(), - runTaskQueue(sideTasks, sideConcurrency), - ]); - } - }); - - if (hostSchedule.remoteArtifactContracts) { - const contractWorker = workers.find( - (worker) => worker.lane === onceLane && worker.shard === 1, - ); - if (!contractWorker) throw new Error("could not select a host artifact contract worker"); - console.log("\nTesting host artifact contracts in a Linux Sandbox..."); - await execIn( - contractWorker, - "pnpm", - [ - "test", - "--reporter=dot", - "-t", - hostInvariantContractPattern, - ...hostInvariantContractFiles, - ], - { - CI: "1", - ASAN_OPTIONS: "detect_leaks=0", - SCRIPTC_TEST_WORKERS: fileWorkers, - }, - "host artifact contracts", - ); - } - })(); - - const local = values["remote-only"] - ? Promise.resolve() - : (async () => { - const hostName = - process.platform === "darwin" - ? "Darwin" - : process.platform === "linux" - ? "Linux" - : process.platform; - const localWork = [ - ...(nativeHostInvariantFiles.length > 0 - ? [`${nativeHostInvariantFiles.length} ${hostName}-native files`] - : []), - ...(hostSchedule.darwinContracts - ? ["compact Darwin platform contracts"] - : []), - ...(hostSchedule.localArtifactContracts - ? ["compact host artifact contracts"] - : []), - `${localLaneFiles.length + localCaseShardedFiles.length} external suites`, - ]; - console.log( - `\nBuilding and testing ${localWork.join(", ")} locally...`, - ); - await run("pnpm", ["build"], { label: "local build" }); - const nativeHostTasks = - nativeHostInvariantFiles.length === 0 - ? [] - : [ - run( - "pnpm", - [ - "test", - ...nativeHostInvariantFiles, - ], - { - env: { - ...sandboxLaneEnv(onceLane), - SCRIPTC_TEST_WORKERS: localTestWorkers, - }, - label: `local ${onceLane} ${hostName}`, - }, - ), - ]; - const artifactContractTasks = - !hostSchedule.localArtifactContracts - ? [] - : [ - run( - "pnpm", - [ - "test", - "--reporter=dot", - "-t", - hostInvariantContractPattern, - ...hostInvariantContractFiles, - ], - { - env: { - ...sandboxLaneEnv(onceLane), - ...(process.platform === "linux" - ? { ASAN_OPTIONS: "detect_leaks=0" } - : {}), - SCRIPTC_TEST_WORKERS: localTestWorkers, - }, - label: `local ${onceLane} ${hostName} artifact contract`, - }, - ), - ]; - const darwinContractTasks = - !hostSchedule.darwinContracts - ? [] - : lanes.map((lane) => - run( - "pnpm", - [ - "test", - "--reporter=dot", - "-t", - hostLaneContractPattern, - ...hostLaneContractFiles, - ], - { - env: { - ...sandboxLaneEnv(lane), - SCRIPTC_TEST_WORKERS: localTestWorkers, - }, - label: `local ${lane} Darwin contract`, - }, - ), - ); - const laneFileTasks = lanes.map((lane) => - run( - "pnpm", - ["test", "--reporter=dot", ...localLaneFiles], - { - env: { - ...sandboxLaneEnv(lane), - SCRIPTC_TEST_RUN_ID: nonce, - SCRIPTC_TEST_WORKERS: "1", - }, - label: `local ${lane} external files`, - }, - ), - ); - const caseTasks = lanes.flatMap((lane) => - Array.from({ length: localCaseShardCount }, (_, offset) => { - const shard = offset + 1; - return run( - "pnpm", - ["test", "--reporter=dot", ...localCaseShardedFiles], - { - env: { - ...sandboxLaneEnv(lane), - SCRIPTC_TEST_SHARD: `${shard}/${localCaseShardCount}`, - SCRIPTC_TEST_RUN_ID: nonce, - SCRIPTC_TEST_WORKERS: "1", - }, - label: `local ${lane} external ${shard}/${localCaseShardCount}`, - }, - ); - }), - ); - const results = await Promise.allSettled( - [ - ...nativeHostTasks, - ...artifactContractTasks, - ...darwinContractTasks, - ...laneFileTasks, - ...caseTasks, - ], - ); - const failures = results.filter((result) => result.status === "rejected"); - if (failures.length) { - for (const result of failures) console.error(result.reason); - throw new Error(`${failures.length} local test lane${failures.length === 1 ? "" : "s"} failed`); - } - })(); - - const results = await Promise.allSettled([remote, local]); - const failures = results.filter((result) => result.status === "rejected"); - if (failures.length) { - for (const result of failures) console.error(result.reason); - throw new Error(`${failures.length} test path${failures.length === 1 ? "" : "s"} failed`); - } - - console.log( - `\n${lanes.length === 2 ? "Both test lanes" : `${lanes[0]} test lane`} passed in ${((Date.now() - suiteStarted) / 60_000).toFixed(1)} minutes.`, - ); -} catch (error) { - failure = error; -} finally { - await cleanup(); - await rm(temp, { recursive: true, force: true }); - if (values.keep && created.size) { - console.log(`Kept sandboxes:\n${[...created].map((name) => ` ${name}`).join("\n")}`); - } -} - -if (failure) throw failure; diff --git a/tests/harness/runtime-tree-shaking.test.ts b/tests/harness/runtime-tree-shaking.test.ts index d991fd98a7..dcde5fe6aa 100644 --- a/tests/harness/runtime-tree-shaking.test.ts +++ b/tests/harness/runtime-tree-shaking.test.ts @@ -15,12 +15,11 @@ import { compile } from "@scriptc/compiler"; const execFileAsync = promisify(execFile); const repoRoot = join(import.meta.dirname, "../.."); const cacheDir = join(repoRoot, "node_modules/.cache/scriptc-tests/runtime-tree-shaking"); -// These source-toolchain contracts are safe in the Linux Sandboxes used by -// `test:sandbox`: they deliberately use the C backend and `nm`, both of -// which are part of that image. Keep the Windows host out of this POSIX -// fixture (its child program uses /bin/echo), but do not use -// SCRIPTC_PORTABLE_ONLY here: that marker means "run in a Linux Sandbox", -// not "skip native executable assertions". +// These source-toolchain contracts deliberately use the C backend and `nm`. +// Keep the Windows host out of this POSIX fixture (its child program uses +// /bin/echo), but do not use SCRIPTC_PORTABLE_ONLY here: that marker means +// "skip host-specific native contracts", not "skip native executable +// assertions". const sourceToolchainTest = process.platform === "win32" ? test.skip : test; interface Fixture { diff --git a/tests/harness/sandbox-command.test.ts b/tests/harness/sandbox-command.test.ts deleted file mode 100644 index 6c1ab796be..0000000000 --- a/tests/harness/sandbox-command.test.ts +++ /dev/null @@ -1,78 +0,0 @@ -import { execFile, execFileSync } from "node:child_process"; -import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; -import { join } from "node:path"; -import { promisify } from "node:util"; -import { afterEach, expect, test } from "vitest"; -import { REMOTE_COMMAND_PENDING, sandboxCommand, sandboxStatusCommand } from "../../scripts/sandbox-command.mjs"; - -const roots: string[] = []; -const statuses: string[] = []; - -afterEach(async () => { - await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true }))); - await Promise.all(statuses.splice(0).map((path) => rm(path, { force: true }))); -}); - -test("short commands stay inline and retain the remote exit contract", async () => { - const marker = `__SCRIPTC_COMMAND_TEST_${process.pid}_SHORT__`; - const prepared = sandboxCommand("sh", ["-c", "exit 7"], marker); - statuses.push(prepared.statusPath); - - expect(prepared.file).toBe(false); - const output = execFileSync(prepared.argv[0], prepared.argv.slice(1), { encoding: "utf8" }); - expect(output).toBe(`\n${marker}7\n`); - expect(await readFile(prepared.statusPath, "utf8")).toBe("7\n"); -}); - -test("uploaded scripts preserve long and shell-sensitive argument bytes", async () => { - const root = await mkdtemp("/tmp/scriptc-sandbox-command-test-"); - roots.push(root); - const sentinel = join(root, "unexpected"); - const args = [ - "x".repeat(2048), - "quote' and spaces", - `$(touch ${sentinel})`, - "backtick` dollar$ slash\\ newline\nend", - "é".repeat(512), - ]; - const marker = `__SCRIPTC_COMMAND_TEST_${process.pid}_LONG__`; - const prepared = sandboxCommand("printf", ["<%s>\n", ...args], marker); - statuses.push(prepared.statusPath); - expect(prepared.file).toBe(true); - expect(prepared.argv).toEqual(["sh", prepared.scriptPath]); - expect(prepared.argv.every((arg) => arg.length < 128)).toBe(true); - - const localScript = join(root, "command.sh"); - await writeFile(localScript, prepared.script); - const output = execFileSync("sh", [localScript], { encoding: "utf8" }); - expect(output).toBe(args.map((arg) => `<${arg}>\n`).join("") + `\n${marker}0\n`); - await expect(readFile(sentinel)).rejects.toMatchObject({ code: "ENOENT" }); - expect(await readFile(prepared.statusPath, "utf8")).toBe("0\n"); -}); - -test("status recovery waits for a disconnected command's actual exit", async () => { - const root = await mkdtemp("/tmp/scriptc-sandbox-status-test-"); - roots.push(root); - const path = join(root, "command.status"); - const marker = "__STATUS_TEST__"; - const pending = promisify(execFile)("sh", ["-c", sandboxStatusCommand(path, marker, 2)]); - await new Promise((resolve) => setTimeout(resolve, 50)); - await writeFile(path, "7\n"); - expect((await pending).stdout).toBe(`\n${marker}7\n`); - - await writeFile(path, "0\n"); - expect(execFileSync("sh", ["-c", sandboxStatusCommand(path, marker, 0)], { encoding: "utf8" })).toBe(`\n${marker}0\n`); - await writeFile(path, "125\n"); - expect(execFileSync("sh", ["-c", sandboxStatusCommand(path, marker, 0)], { encoding: "utf8" })).toBe(`\n${marker}125\n`); -}); - -test("an absent or incomplete status remains pending rather than passing", async () => { - const root = await mkdtemp("/tmp/scriptc-sandbox-status-test-"); - roots.push(root); - const path = join(root, "command.status"); - const marker = "__STATUS_TEST__"; - const probe = () => execFileSync("sh", ["-c", sandboxStatusCommand(path, marker, 0)], { encoding: "utf8" }); - expect(probe()).toBe(`\n${marker}${REMOTE_COMMAND_PENDING}\n`); - await writeFile(path, ""); - expect(probe()).toBe(`\n${marker}${REMOTE_COMMAND_PENDING}\n`); -}); diff --git a/tests/harness/sandbox-config.test.ts b/tests/harness/sandbox-config.test.ts deleted file mode 100644 index 6f9f3b37bb..0000000000 --- a/tests/harness/sandbox-config.test.ts +++ /dev/null @@ -1,269 +0,0 @@ -import { readFileSync } from "node:fs"; -import { expect, test } from "vitest"; -import { - defaultSandboxImage, - sandboxBootstrapCommand, - sandboxImageConfig, - sandboxRunnerConfig, - sandboxTestWorkerAllocation, - sandboxVcrConfig, - sandboxVercelConfig, - sandboxVercelEnvironment, -} from "../../scripts/sandbox-config.mjs"; - -const oidcToken = (claims: object) => - ["header", Buffer.from(JSON.stringify(claims)).toString("base64url"), "signature"].join("."); - -test("runner settings are read after loading the sandbox environment", () => { - expect( - sandboxRunnerConfig({ - SCRIPTC_SANDBOX_VCPUS: "16", - SCRIPTC_TEST_WORKERS: "7", - SCRIPTC_LOCAL_TEST_WORKERS: "3", - SCRIPTC_LOCAL_CASE_SHARDS: "4", - SCRIPTC_SANDBOX_TIMEOUT: "90m", - }), - ).toEqual({ - vcpus: "16", - testWorkers: "7", - localTestWorkers: "3", - localCaseShards: "4", - sandboxTimeout: "90m", - }); -}); - -test("runner settings retain their documented defaults", () => { - expect(sandboxRunnerConfig({})).toEqual({ - vcpus: "8", - testWorkers: "2", - localTestWorkers: "2", - localCaseShards: "2", - sandboxTimeout: "45m", - }); -}); - -test("the managed Sandbox image is the default", () => { - expect(sandboxImageConfig({})).toEqual({ - custom: false, - reference: defaultSandboxImage, - sandboxImage: "vercel/sandbox/universal", - }); -}); - -test("the managed image bootstraps the ScriptC native toolchain and workspace", () => { - const bootstrap = sandboxBootstrapCommand(false); - expect(bootstrap?.prepareWorkspace).toMatchObject({ command: "sh", workdir: "/vercel" }); - expect(bootstrap?.prepareWorkspace.args.join(" ")).toContain("mkdir -p /workspace"); - expect(bootstrap?.install).toEqual({ - args: ["scripts/sandbox-bootstrap.sh"], - command: "sh", - workdir: "/workspace", - }); - expect(sandboxBootstrapCommand(true)).toBeUndefined(); -}); - -test("the managed-image bootstrap pins the custom image's Node, pnpm, and LLVM", () => { - const bootstrap = readFileSync(new URL("../../scripts/sandbox-bootstrap.sh", import.meta.url), "utf8"); - expect(bootstrap).toContain("< .node-version"); - expect(bootstrap).toContain("ARG PNPM_VERSION="); - expect(bootstrap).toContain("llvm-toolchain-${llvm_distro}-22"); - expect(bootstrap).toContain("install -D -m 0644"); - expect(bootstrap).toContain("clang-22"); - expect(bootstrap).toContain("libclang-rt-22-dev"); - expect(bootstrap).toContain("llvm-22-dev"); - expect(bootstrap).toContain("ninja-build"); -}); - -test("a custom image does not select the Sandbox team or project", () => { - expect( - sandboxImageConfig({ - SCRIPTC_SANDBOX_IMAGE: "vcr.vercel.com/image-team/image-project/scriptc-tests:node24", - }), - ).toEqual({ - custom: true, - reference: "vcr.vercel.com/image-team/image-project/scriptc-tests:node24", - repository: "scriptc-tests", - sandboxImage: "vcr.vercel.com/image-team/image-project/scriptc-tests:node24", - tag: "node24", - }); -}); - -test("custom images must be fully qualified VCR references", () => { - expect(() => sandboxImageConfig({ SCRIPTC_SANDBOX_IMAGE: "scriptc-tests:node24" })).toThrow( - "fully qualified VCR image", - ); -}); - -test("OIDC authentication wins and supplies its own Sandbox scope", () => { - const config = sandboxVercelConfig({ - VERCEL_OIDC_TOKEN: "header.payload.signature", - VERCEL_TOKEN: "legacy-token", - }); - expect(config).toMatchObject({ - authSource: "VERCEL_OIDC_TOKEN", - authToken: "header.payload.signature", - oidc: true, - scopeArgs: [], - scopeSource: "VERCEL_OIDC_TOKEN claims", - }); - expect( - sandboxVercelEnvironment(config, { - VERCEL_OIDC_TOKEN: "header.payload.signature", - VERCEL_TOKEN: "legacy-token", - }), - ).toEqual({ - VERCEL_AUTH_TOKEN: "header.payload.signature", - VERCEL_OIDC_TOKEN: "header.payload.signature", - }); -}); - -test("OIDC scope wins over complete or partial legacy environment scope", () => { - for (const legacyScope of [ - { VERCEL_PROJECT_ID: "stale-project", VERCEL_TEAM_ID: "stale-team" }, - { VERCEL_TEAM_ID: "stale-team" }, - ]) { - expect( - sandboxVercelConfig({ - ...legacyScope, - VERCEL_OIDC_TOKEN: "header.payload.signature", - }), - ).toMatchObject({ - authSource: "VERCEL_OIDC_TOKEN", - oidc: true, - project: undefined, - scopeArgs: [], - scopeSource: "VERCEL_OIDC_TOKEN claims", - team: undefined, - }); - } -}); - -test("access-token authentication uses explicit environment scope", () => { - expect( - sandboxVercelConfig({ - VERCEL_PROJECT_ID: "prj_example", - VERCEL_TEAM_ID: "team_example", - VERCEL_TOKEN: "legacy-token", - }), - ).toMatchObject({ - authSource: "VERCEL_TOKEN", - authToken: "legacy-token", - oidc: false, - project: "prj_example", - scopeArgs: ["--scope", "team_example", "--project", "prj_example"], - scopeSource: "VERCEL_TEAM_ID + VERCEL_PROJECT_ID", - team: "team_example", - }); -}); - -test("VCR uses an access-token fallback with scope decoded from OIDC claims", () => { - const token = oidcToken({ owner_id: "team_oidc", project_id: "prj_oidc" }); - const config = sandboxVercelConfig({ - VERCEL_OIDC_TOKEN: token, - VERCEL_PROJECT_ID: "stale-project", - VERCEL_TEAM_ID: "stale-team", - VERCEL_TOKEN: "registryaccesstoken", - }); - - expect( - sandboxVcrConfig(config, { - VERCEL_AUTH_TOKEN: "stale-auth-token", - VERCEL_OIDC_TOKEN: token, - VERCEL_TOKEN: "registryaccesstoken", - }), - ).toEqual({ - authSource: "VERCEL_TOKEN", - env: { - VERCEL_OIDC_TOKEN: token, - VERCEL_TOKEN: "registryaccesstoken", - }, - scopeArgs: ["--scope", "team_oidc", "--project", "prj_oidc"], - scopeSource: "VERCEL_OIDC_TOKEN claims", - }); -}); - -test("VCR never passes an OIDC JWT through VERCEL_TOKEN", () => { - const token = oidcToken({ owner_id: "team_oidc", project_id: "prj_oidc" }); - const config = sandboxVercelConfig({ - VERCEL_OIDC_TOKEN: token, - VERCEL_TOKEN: token, - }); - - expect( - sandboxVcrConfig(config, { - VERCEL_AUTH_TOKEN: token, - VERCEL_OIDC_TOKEN: token, - VERCEL_TOKEN: token, - }), - ).toEqual({ - authSource: "Vercel CLI login", - env: { VERCEL_OIDC_TOKEN: token }, - scopeArgs: ["--scope", "team_oidc", "--project", "prj_oidc"], - scopeSource: "VERCEL_OIDC_TOKEN claims", - }); -}); - -test("VCR uses the selected access token and explicit environment scope", () => { - const config = sandboxVercelConfig({ - VERCEL_AUTH_TOKEN: "accesstoken", - VERCEL_PROJECT_ID: "prj_example", - VERCEL_TEAM_ID: "team_example", - }); - - expect(sandboxVcrConfig(config, { VERCEL_AUTH_TOKEN: "accesstoken" })).toEqual({ - authSource: "VERCEL_AUTH_TOKEN", - env: { VERCEL_TOKEN: "accesstoken" }, - scopeArgs: ["--scope", "team_example", "--project", "prj_example"], - scopeSource: "VERCEL_TEAM_ID + VERCEL_PROJECT_ID", - }); -}); - -test("VCR rejects JWT-shaped access tokens when OIDC is not selected", () => { - const token = oidcToken({ owner_id: "team_example", project_id: "prj_example" }); - const config = sandboxVercelConfig({ - VERCEL_AUTH_TOKEN: token, - VERCEL_PROJECT_ID: "prj_example", - VERCEL_TEAM_ID: "team_example", - }); - - expect(() => sandboxVcrConfig(config, { VERCEL_AUTH_TOKEN: token })).toThrow( - "is not a VCR-compatible access token", - ); -}); - -test("VCR rejects OIDC tokens without usable project claims", () => { - const token = oidcToken({ owner_id: "team_oidc" }); - const config = sandboxVercelConfig({ VERCEL_OIDC_TOKEN: token }); - - expect(() => sandboxVcrConfig(config, { VERCEL_OIDC_TOKEN: token })).toThrow( - "owner_id and project_id claims", - ); -}); - -test("access-token scope failures explain every required variable", () => { - expect(() => sandboxVercelConfig({ VERCEL_TOKEN: "legacy-token" })).toThrow( - "VERCEL_TOKEN requires VERCEL_TEAM_ID and VERCEL_PROJECT_ID", - ); - expect(() => sandboxVercelConfig({ VERCEL_TEAM_ID: "team_example" })).toThrow( - "VERCEL_TEAM_ID and VERCEL_PROJECT_ID must be set together", - ); -}); - -test("test processes stay within the per-Sandbox worker budget", () => { - expect(sandboxTestWorkerAllocation(4, 1)).toEqual({ - caseWorkers: 3, - sideConcurrency: 1, - }); - expect(sandboxTestWorkerAllocation(4, 2)).toEqual({ - caseWorkers: 2, - sideConcurrency: 2, - }); - expect(sandboxTestWorkerAllocation(2, 2)).toEqual({ - caseWorkers: 1, - sideConcurrency: 1, - }); - expect(sandboxTestWorkerAllocation(1, 2)).toEqual({ - caseWorkers: 1, - sideConcurrency: 0, - }); -}); diff --git a/tests/harness/sandbox-dockerfile.test.ts b/tests/harness/sandbox-dockerfile.test.ts deleted file mode 100644 index 298087410e..0000000000 --- a/tests/harness/sandbox-dockerfile.test.ts +++ /dev/null @@ -1,23 +0,0 @@ -import { readFileSync } from "node:fs"; -import { describe, expect, test } from "vitest"; - -const dockerfile = readFileSync(new URL("../../Dockerfile.sandbox", import.meta.url), "utf8"); -const dockerignore = readFileSync(new URL("../../.dockerignore", import.meta.url), "utf8"); - -describe("Sandbox Dockerfile", () => { - test("copies CLI lifecycle scripts before installing workspace dependencies", () => { - const packageIndex = dockerfile.indexOf( - "COPY packages/cli/package.json packages/cli/package.json", - ); - const scriptIndex = dockerfile.indexOf( - "COPY packages/cli/scripts/warm-cache.mjs packages/cli/scripts/warm-cache.mjs", - ); - const installIndex = dockerfile.indexOf("RUN pnpm install --frozen-lockfile"); - - expect(packageIndex).toBeGreaterThanOrEqual(0); - expect(scriptIndex).toBeGreaterThan(packageIndex); - expect(installIndex).toBeGreaterThan(scriptIndex); - expect(dockerignore).toContain("!packages/cli/scripts/"); - expect(dockerignore).toContain("!packages/cli/scripts/warm-cache.mjs"); - }); -}); diff --git a/tests/harness/sandbox-platform.test.ts b/tests/harness/sandbox-platform.test.ts deleted file mode 100644 index 78c5e62550..0000000000 --- a/tests/harness/sandbox-platform.test.ts +++ /dev/null @@ -1,52 +0,0 @@ -import { expect, test } from "vitest"; -import { - sandboxHostSchedule, - sandboxLaneEnv, -} from "../../scripts/sandbox-platform.mjs"; - -const files = ["native-a.test.ts", "native-b.test.ts"]; - -test("Darwin runs native files and kqueue contracts locally", () => { - expect(sandboxHostSchedule("darwin", files)).toEqual({ - darwinContracts: true, - localArtifactContracts: true, - localInvariantFiles: files, - remoteArtifactContracts: false, - remoteInvariantFiles: [], - }); -}); - -test("Linux runs its supported native files locally without Darwin contracts", () => { - expect(sandboxHostSchedule("linux", files)).toEqual({ - darwinContracts: false, - localArtifactContracts: true, - localInvariantFiles: files, - remoteArtifactContracts: false, - remoteInvariantFiles: [], - }); -}); - -test("other hosts retain native-file coverage in Linux Sandboxes", () => { - expect(sandboxHostSchedule("win32", files)).toEqual({ - darwinContracts: false, - localArtifactContracts: false, - localInvariantFiles: [], - remoteArtifactContracts: true, - remoteInvariantFiles: files, - }); -}); - -test("lane environments override inherited sanitizer and scheduler settings", () => { - expect(sandboxLaneEnv("plain")).toEqual({ - CI: "1", - SCRIPTC_PORTABLE_ONLY: "", - SCRIPTC_SAN: "", - SCRIPTC_TEST_SHARD: "", - }); - expect(sandboxLaneEnv("san")).toEqual({ - CI: "1", - SCRIPTC_PORTABLE_ONLY: "", - SCRIPTC_SAN: "1", - SCRIPTC_TEST_SHARD: "", - }); -}); diff --git a/tests/test262/README.md b/tests/test262/README.md index bc5d984429..c1255f5917 100644 --- a/tests/test262/README.md +++ b/tests/test262/README.md @@ -10,7 +10,7 @@ pnpm test:test262 The vendored inputs are regression fixtures for developing the runner and compiler. Strict variants are checked against `expectations.json`; entries without an expectation require successful completion. Sloppy variants remain visible as exclusions. A recorded refusal remains a refusal, and a different outcome fails the regression gate. An external snapshot survey treats every executed non-pass as a failing exit status. An all-excluded selection also fails, except for empty distributed shards. -The same regression tests and host assertion checks are included in `pnpm test`. `pnpm test:sandbox` partitions the cases across both plain and sanitized lanes. The default backend may fall back from LLVM to C according to the ordinary compiler policy; every executed result records the actual backend. `--backend llvm` or `--backend c` pins a backend, and `SCRIPTC_SAN=1` enables sanitizers. +The same regression tests and host assertion checks are included in `pnpm test`. The plain and sanitized lanes both cover these cases. The default backend may fall back from LLVM to C according to the ordinary compiler policy; every executed result records the actual backend. `--backend llvm` or `--backend c` pins a backend, and `SCRIPTC_SAN=1` enables sanitizers. ## Full snapshot surveys