Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/build-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -467,6 +467,10 @@ jobs:
retention-days: 7

package-ipk:
# Frugality: SDK packaging is release-critical, not PR-blocking — PRs run the
# fast path (compile + happy-path + contract lanes); the matrix runs on
# push-to-main and tags. Override via workflow_dispatch.
if: github.event_name != 'pull_request'
timeout-minutes: 30
needs: [define-package-matrix, determine-versioning, compile-binaries, build-portal]
runs-on: ubuntu-latest
Expand Down Expand Up @@ -660,6 +664,10 @@ jobs:
echo "Uploaded $PACKAGE_FILENAME → $PKG_HASH"

package-apk:
# Frugality: SDK packaging is release-critical, not PR-blocking — PRs run the
# fast path (compile + happy-path + contract lanes); the matrix runs on
# push-to-main and tags. Override via workflow_dispatch.
if: github.event_name != 'pull_request'
timeout-minutes: 90
needs: [define-package-matrix, determine-versioning, compile-binaries, build-portal]
runs-on: ubuntu-latest
Expand Down
90 changes: 90 additions & 0 deletions .github/workflows/count-soak.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Count soak — the state-leak family gate (#852).
#
# WHY THIS LANE EXISTS: every other PR-gating Go lane runs -count=1,
# which is structurally blind to the state-leak family — package-level
# state in src/merchant / src/valve that outlives the test which mutated
# it. The signature, pinned: FIRST-ITERATION-GREEN. Iteration 1 of the
# test binary passes and later iterations fail, because -count=N re-runs
# a package's tests inside ONE process where package vars, maps and
# leaked goroutines persist across iterations. #821/#732/#841/#845 were
# all found by hand soaks; #821 sat red on main for days while every
# -count=1 lane stayed green.
#
# THE SPLIT — every count lane that exists after this change:
# - THIS lane (push + PR; ngit-ci executes it on every PR): the merge
# gate. Catches the deterministic members (iteration 2 fails
# immediately) at merge time — a red CI row instead of a three-issue
# investigation.
# - nightly-stress.yml (#697): the deep soak — merchant+valve+tollwallet
# at -count=5, full scope, where probabilistic members (the #845
# 1-in-8 signature) surface over repeated runs. ngit-ci ignores
# `schedule:` (.ngit/README.md), so that file is manual-replay only
# today; THIS lane is the one that actually gates merges.
#
# WHY ITS OWN FILE, not a job in test.yml: ngit-ci bounds a whole act
# invocation at 1800 s (.ngit/README.md, "Sharded stage 2") and caps a
# job at 2 CPUs. The merchant suite measures ~300 s per iteration under
# -race (measured 2026-10-12: count=5 = 1503 s on a 16-core host, so
# ~20 min on the runner class is optimistic), so -count=5 cannot share
# test.yml's invocation budget. One file = one budget, the same doctrine
# the release shards follow.
#
# COUNTS, per module, from measurement (2026-10-12, clean worktree of
# f89c879a, go1.26.8, 16-core host, warm build cache; -count!=1 disables
# Go's test-result caching, so this lane pays full runtime on every run):
# src/valve -count=5 : GREEN in 47.1 s — cheap, so it soaks deeper;
# src/merchant -count=2 : GREEN in 597 s (~10 min here, budget ~20 min
# on a 2-CPU-capped runner) — the deterministic
# members fail on iteration 2 (planted-drift
# proof in the #852 PR: reverting #842's cleanup
# fix makes this exact command exit 1 with the
# victim failing in 0.01 s, while a -count=1 run
# of the same tree stays green in 314 s);
# -count=5 (measured 1503 s) is the nightly
# lane's job, not the PR lane's budget.
#
# THE SKIP (temporary — remove when #846 lands): #845 (owed-grant
# monitors and the late-receive recorder exec'ing ndsctl out of their
# tests' PATH windows, probabilistic ~1-in-8) is unfixed on main while
# its fix rides OPEN PR #846. Without the skip this lane would flake red
# ~1-in-8 and train re-run-and-ignore. Deleting the -skip when #846
# merges is the lane going fully red-catching — tracked in the #852 PR
# body.
#
# VERIFICATION RULE for every fix in this family: the PR must include a
# -count>=5 soak of the offender pair in its verification (the #837/#846
# pattern), or the next member is another three-issue investigation.
name: Count soak

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

on:
push:
branches: [main, master, develop]
pull_request:
branches: [main, master, develop]

jobs:
count-soak:
timeout-minutes: 35
runs-on: ubuntu-latest
strategy:
# Both modules report even when one is red: the pair is the
# signal, not the first failure.
fail-fast: false
matrix:
include:
- module: src/valve
soak-flags: "-count=5 -timeout=10m"
- module: src/merchant
soak-flags: "-count=2 -timeout=25m -skip '^TestFirstTimePurchaseStillRefusedWhenNdsDoesNotKnowTheClient$'"
steps:
- uses: actions/checkout@v6
- uses: actions/setup-go@v6
with:
go-version-file: src/go.mod
- name: Count soak (-race, per-module count)
working-directory: ${{ matrix.module }}
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-test-summary.sh" "count-soak (${{ matrix.module }})" -v -race -tags testenv ${{ matrix.soak-flags }} .
3 changes: 2 additions & 1 deletion .github/workflows/gitleaks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ on:
branches: [main]
pull_request:
schedule:
- cron: "13 6 * * *"
- cron: "13 6 * * 1"

jobs:
# Frugality: weekly (was daily) — pre-commit hook + PR lane carry daily coverage
gitleaks:
uses: Amperstrand/.github/.github/workflows/gitleaks.yml@main
6 changes: 6 additions & 0 deletions .github/workflows/repro-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ name: Reproducibility check
# packaging/build-inputs.json at runtime so the manifest stays the single
# source of truth.

concurrency:
group: repro-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

on:
push:
branches: [main]
Expand All @@ -24,6 +28,7 @@ on:

jobs:
binaries:
timeout-minutes: 15
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
Expand All @@ -48,6 +53,7 @@ jobs:
run: bash scripts/repro-test.sh binaries x86_64

packages:
timeout-minutes: 45
if: github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
needs: binaries
Expand Down
27 changes: 27 additions & 0 deletions .github/workflows/selfhosted-probe.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: selfhosted-probe

# Proves the ai-legion-tbmg self-hosted runner end-to-end (registration,
# pickup, docker access, toolchain fetch) before any release-critical job
# routes to it. Manual-only until proven; then the recurring heavy lanes
# (path-parity weekly, repro packages on tags) migrate in a follow-up.
on:
workflow_dispatch:

jobs:
probe:
runs-on: [self-hosted, tbmg]
timeout-minutes: 10
steps:
- name: Runner identity
run: |
echo "runner: $(hostname) | $(uname -m) | $(nproc) cores"
docker version --format 'docker: {{.Server.Version}}'
- uses: actions/checkout@v4
- name: Manifest toolchain fetch (the parity pattern)
run: |
GOV=$(jq -r '.go.version' packaging/build-inputs.json)
GOURL=$(jq -r '.go.tarball_linux_amd64.url' packaging/build-inputs.json)
echo "expected go: $GOV"
curl -sSfL --retry 3 -m 120 "$GOURL" -o /tmp/probe-go.tgz && ls -la /tmp/probe-go.tgz
- name: Report
run: echo "probe OK — self-hosted lane proven $(date -u +%FT%TZ)"
9 changes: 9 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
name: Test

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

on:
push:
branches: [main, master, develop]
Expand All @@ -8,6 +12,7 @@ on:

jobs:
go-test:
timeout-minutes: 20
runs-on: ubuntu-latest
strategy:
# Fail fast: one red module lane stops the run instead of burning the
Expand Down Expand Up @@ -46,6 +51,7 @@ jobs:
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-test-summary.sh" "${{ matrix.module }}" ./... -v -count=1 -race

main-test:
timeout-minutes: 15
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
Expand All @@ -57,6 +63,7 @@ jobs:
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-test-summary.sh" "main package (testenv)" -v -count=1 -race -tags testenv .

contract-lint:
timeout-minutes: 5
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
Expand Down Expand Up @@ -305,6 +312,7 @@ jobs:
greatspectate check --config specquotes.toml --comment-start "// " --comment-continue "//" $FILES

build-purity:
timeout-minutes: 10
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
Expand All @@ -327,6 +335,7 @@ jobs:
bash tests/contract/check-ssid-naming.sh

deps-and-imports:
timeout-minutes: 5
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
Expand Down
3 changes: 2 additions & 1 deletion .ngit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ untouched; the two systems run side by side.
| --- | --- |
| `act/workflows/test.yml` | The port of `.github/workflows/test.yml`: per-module Go tests over the module matrix, the main-package `testenv` test, `js-schema-lint`, the `Spec-quote drift check`, `build-purity`, and the dependency/import-path checks. |
| `act/workflows/go-test.yml` | The pre-PR sequence documented in [AGENTS.md](../AGENTS.md), run from `src/`: `gofmt -l .`, `go vet ./...`, `go build ./...`, `go test -race -count=1 -tags testenv ./...`. |
| `act/workflows/count-soak.yml` | The state-leak family gate (#852), push/PR-triggered: `-race` count soaks of the two stateful offenders (`src/valve` at `-count=5`, `src/merchant` at `-count=2` with a temporary `-skip` until #846 lands) — every other PR-gating lane runs `-count=1`, which cannot see package state that outlives its test. Own file because ngit-ci bounds an act invocation at 1800 s; the deep `-count=5` soak stays in `nightly-stress.yml`. |
| `act/workflows/repro-check.yml` | The fast lane of `.github/workflows/repro-check.yml`: rebuild both Go binaries in two independent clean roots (separate HOME, module and build caches) and require byte-identical SHA-256s. The package targets (`portal`, `ipk`, `ipk-upx`, `apk`) stay on the GitHub workflow's `workflow_dispatch` slow lane and on a build host (they need `docker`), and are covered here by `build-package*.yml` below. |
| `act/workflows/build-package-binaries.yml` | Stage 1 of the release pipeline: cross-compile the five GOARCH/GOARM/GOMIPS targets, build the captive-portal assets, mirror both to Blossom, and publish the build-id records stage 2 consumes. |
| `act/workflows/build-package-<shard>.yml` | Stage 2, **sharded**: one workflow file per group of legs, rendered from [`packaging/ngit-release-matrix.json`](../packaging/ngit-release-matrix.json) by [`scripts/ngit-gen-shards.py`](../scripts/ngit-gen-shards.py). Eleven shards cover the same 14 `.ipk` + 3 `.apk` matrix the single file used to. Each builds its legs, mirrors them to Blossom, and publishes one kind-30078 record per leg — **no** kind-1063. See "Sharded stage 2" below. |
| `act/workflows/build-package-announce.yml` | The only file that publishes kind-1063. It runs [`scripts/ngit-release-announce.sh`](../scripts/ngit-release-announce.sh), which refuses unless every shard of this (version, channel, release run) reported success and every leg in the plan has a build record naming that same release run; then it announces, then it runs the publication gate over the whole matrix. |
| `act/workflows/build-package-announce.yml` | The only file that publishes kind-1063. Its `happy-path` job first exercises the behavioral gate on the exact x86_64 `.apk` this release built (fetched from the leg's own kind-30078 build record, sha256-verified against it), and `announce` needs it — a red suite skips the announcement, fail-closed. It then runs [`scripts/ngit-release-announce.sh`](../scripts/ngit-release-announce.sh), which refuses unless every shard of this (version, channel, release run) reported success and every leg in the plan has a build record naming that same release run; then it announces, then it runs the publication gate over the whole matrix. |
| `act/workflows/verify-publication.yml` | The publication gate on its own: for every `(arch, format)` the release matrix declares, a kind-1063 announcement must exist for the verified version+channel, and the artifact must be fetchable from >= 2 Blossom mirrors with the sha256 carried in its `x` tag. Started by a `verify/<version>/<channel>[/<scope>]` ref, which names the published version to audit. Expectations are the **union** of the shard files. |

Two Go files because they cover different things: `test.yml` tests the nested
Expand Down
Loading