[Amperstrand]

What

tests/installer-contract/run.sh + README: the installer validation performed by hand on the vlab lane on 2026-10-11, made repeatable — install a built .apk on a pristine OpenWrt 25.12 VM with the documented operator env (TOLLGATE_ADMIN_PASSWORD, TOLLGATE_PRIVATE_WIFI_PASSWORD), then assert the credential and convergence contracts, including reboot persistence. Output idiom matches tests/happy-path (ICCHECK/ICRESULT/ICEXIT).

Verified end-to-end on the vlab lane (ai-legion, OpenWrt 25.12.5 x86-64, tollgate-wrt-0.6.0_rc1-r0.apk @ 362e5d5, sha256 a9a8c007…)

• fresh-boot VM bootstrapped over serial (dhcp fix + dropbear key) when ssh is down — the exact manual plumbing, now built in;
• pristine verification, not trust: firstboot -y && reboot was proven to silently keep /etc on this image (false pristine → credential assertions would lie); the lane forces reboot -f AND asserts the clean state (no tollgate-wrt, empty root credential);
• full run: 25 checks, 22 PASS, 3 FAIL (all designed), 0 unintended:
  • contract:credential-applied FAIL = the installer: TOLLGATE_ADMIN_PASSWORD / TOLLGATE_PRIVATE_WIFI_PASSWORD never reach the uci-defaults script under apk add on 25.12 — the documented install-time credential UX is broken #888 acceptance criterion (apk-tools drops the install-time env; the manual session proved the script honors the env when run directly);
  • contract:provisional-lifecycle PASS = the safe degraded path pinned (silent random credential, marker, FIRST-RUN page);
  • listeners:8090/:8443 FAIL = the entry-port flip: LuCI's :8090/:8443 pair does not converge after a fresh-image install (board mode) — caught by the new installer-contract lane #890 acceptance criterion — the lane caught the entry-port flip's LuCI half not converging on a true fresh-image install (present on the earlier already-booted path; uhttpd main ends with no listeners).
• everything else green: transfer hash, apk closure, no-plaintext, services, the board's own pair (:8080/:443 TLS answers), guest :80, mapping, persistence across reboot.

Why shipped red

The two known-reds are the acceptance criteria of open issues; a fence that cannot fail is decoration. README documents the flip-to-green conditions (#888 fix ⇒ credential-applied green + provisional-lifecycle SKIP; #890 fix ⇒ listeners green).

Out of scope (README): radios/SSID, br-private, ipk-era — physical/PRTA lane ground.

Closes nothing; acceptance-for #888 #890. Review lane: wave2a.

[Amperstrand]

Update: file-channel default + #888 closed. The lane now defaults to --credentials-mode file (#894's staged-secret channel) and gained contract:credentials-consumed, radio-aware for x86 VMs (the wifi secret's reader lives in wireless setup). Verified end-to-end on a fresh-boot pristine VM: credential-applied PASS (sha256crypt match through apk add), consumed-check green under the radio-aware grading, everything else per the README's known-red table (#890 listeners only). The env mode remains as the living demonstration of the apk-tools env loss for #888's record.

[Amperstrand]

Nominated to the v0.6.0 release scope (#909). If the scope holds, this is on the critical path for final — review priority requested. Deferred PRs: no action needed; the scope issue lists you for 0.6.1+.

[Amperstrand]

rc3 contract update + a new stop-ship red. The lane now asserts the rc3 installer contract (head f260f89e, rebased onto rc3): clean preinst refusal on passwordless root, operator-sets-password flow, package-sets-no-credential (hash byte-identical across install), no provisional marker, persistence. The superseded staged-secret checks were dropped (#897/#902/#907 redesign — history in #888).

First rc3 run (vlab, pristine VM, tollgate-wrt-0.6.0_rc3-r0.apk sha256 9fcc15ff…): contract:preinst-refuses-empty-root is RED — and the red is a genuine stop-ship finding, not a lane gap: apk-tools executes the failing pre-install but completes the transaction anyway (package registered, service running, guidance text suppressed, exit 1). Filed as #911 with both live runs' evidence; the lane check is its acceptance criterion. The operator-flow install after setting a root password proceeds correctly, and the rc3 artifact builds clean through the pinned pipeline.

[Amperstrand]

Batch-3 (orchestrator sprint): #892 (installer contract lane) — VM-validated install path for built .apks, the missing install-tier gate. High-value test infra. Merge-ready.

[Amperstrand]

First fully-automated gate run — evidence update.

Run of 2026-10-13 on ai-legion (pristine OpenWrt 25.12.5 x86-64 QEMU VM, snapshot mode, single user-net NIC, serial bootstrap), artifact tollgate-wrt-0.6.0_rc4-r0.apk (sha256 79e67a59…, built from 6b572dd = this branch merged with main tip, via the fail-closed build path of #918):

ICRESULT total=43 pass=31 fail=12 skip=0 not_verified=0
ICEXIT 1

Full transcript + machine-readable verdict committed at
tests/installer-contract/evidence/2026-10-13-ai-legion-rc4/.

Every red is attributed, none loosened:

• listeners:uhttpd-sections / mapping:port-ownership / listeners:8090/8443 / logs:crash-signatures (+ their persistence: twins) — the fix(setup): fresh install left uhttpd.main with no listeners — entry_ui must not wait for the marker its writer has not yet run (#849) #899 tear, live: uhttpd.main ends the install listenerless, admin holds [8080 443], logread carries No sockets bound ×5 pre-reboot / ×2 post-reboot. The section/ownership checks read uci show uhttpd itself, so this is detected even in the stealth variant where another instance claims the ports (pinned offline by the selftest).
• contract:preinst-refuses-empty-root — new shape, not stop-ship: the #907 preinst gate does not refuse installs on the apk era — apk-tools completes the transaction (and starts the service) despite the pre-install failure #911's: on the rc4-era build apk add on an empty-root device exits 0 (guidance suppressed, package registered, service running) — the preinst gate does not fire at all. Consistent with setup: default to a fixed TollGate credential instead of refusing #913 ("default to a fixed TollGate credential instead of refusing").
• contract:no-provisional-marker — same setup: default to a fixed TollGate credential instead of refusing #913 consequence: a credential is now generated on pristine install (marker present).

Maintainer decision needed (assertions deliberately left red): #913 changed the installer contract this lane pins. Either the contract text/assertions get updated to the #913 semantics (fixed default credential + marker expected, preinst expectation dropped or redefined), or #913's interaction with the preinst gate is a regression to fix. The lane reports; it does not decide.

31 greens include the full operator flow (password set → install → hash byte-identical → no plaintext), all reachable listeners, both HTTP planes, mapping, and every persistence check that can hold on this build. Offline selftest: SELFTESTRESULT total=18 ok=18 bad=0.

[Amperstrand]

Tier evidence — docs wave vs main @ f66cd67

Verdict: CLEAN MERGE — lane scripts syntax-clean.

• git merge-tree vs f66cd67: clean; real merge applies cleanly.
• changelog-assemble.py check on the merged tree: no findings; own fragment 892-installer-contract-lane.internal.md — bullet body + PR-link tail verified.
• Content skim: adds tests/installer-contract/ (README, run/checks/selftest, committed evidence from the 2026-10-13 ai-legion rc4 run) — the M2 installer-contract lane.
• bash -n on run.sh, checks.sh, selftest.sh: ok.

Tracker: #965. Scratch worktrees only — no shared refs touched.

[felixfelix-bot]

Review: APPROVE — shipping red is the correct call here, and the redness is itemised

Reviewed cold at head mod-pr892. A new installer contract lane (+1176/-0, 7 files) that makes a
hand-run manual validation repeatable, and ships red on purpose. I ran its self-test and read its
evidence.

The self-test is the strongest signal: 18/18, and every case is paired with its expected defect.

SELFTESTRESULT total=18 ok=18 bad=0
SELFTESTEXIT 0

SELFCASE sections/healthy-board            ok   (PASS)
SELFCASE sections/torn-899-bare-main       ok   (FAIL)
SELFCASE sections/torn-899-stealth         ok   (FAIL)
SELFCASE ownership/board-on-luci-layout    ok   (FAIL)
SELFCASE ownership/unknown-mapping         ok   (FAIL)
SELFCASE crash/clean-log                   ok   (PASS)
SELFCASE crash/no-sockets-bound            ok   (FAIL)
SELFCASE preinst/clean-refusal             ok   (PASS)
SELFCASE preinst/911-completed-transaction ok   (FAIL)
SELFCASE preinst/exit-zero-not-gated       ok   (FAIL)
SELFCASE preinst/refused-but-registered    ok   (FAIL)

The ok/(PASS|FAIL) pairing is the right construction: a synthetic healthy input must be
accepted and its torn counterpart must be refused, so "the check passes" cannot be achieved by a
check that refuses everything. torn-names-main and preinst/911-names-the-shape assert the
diagnostic rather than the verdict — i.e. that the failure names the defect — which is the part
that usually rots silently. This is a lane that proves it can fail, which is what makes its green
results worth anything.

Shipping red is honest, and the evidence says so up front. evidence/2026-10-13-ai-legion-rc4/verdict.json
records the actual run — "summary": {"pass": 31, "fail": 12, "skip": 0, "not_verified": 0, "exit": 1}
against tollgate-wrt-0.6.0_rc4-r0.apk @ 6b572dd on a pristine OpenWrt 25.12.5 x86-64 VM — and the
PR title itself says "shipped red". A lane committed with a truthful failing baseline is far more
useful than one committed green because its checks were weakened to pass; the failures are the
finding, not a defect of the PR.

The known-reds are itemised with root causes, not waved at. The README lists each one and why it
is red — and crucially, why it is asserted anyway: "The remaining known-reds are kept asserted on
purpose":

• installer: TOLLGATE_ADMIN_PASSWORD / TOLLGATE_PRIVATE_WIFI_PASSWORD never reach the uci-defaults script under apk add on 25.12 — the documented install-time credential UX is broken #888's contract:credential-applied — apk-tools does not deliver install-time env to the
  uci-defaults script;
• entry-port flip: LuCI's :8090/:8443 pair does not converge after a fresh-image install (board mode) — caught by the new installer-contract lane #890's listeners:8090 / listeners:8443, plus the config-reading listeners:uhttpd-sections
  / mapping:port-ownership — the fix(setup): fresh install left uhttpd.main with no listeners — entry_ui must not wait for the marker its writer has not yet run (#849) #899 tear (a listenerless uhttpd.main) still present on
  fresh-image installs;
• stop-ship: the #907 preinst gate does not refuse installs on the apk era — apk-tools completes the transaction (and starts the service) despite the pre-install failure #911's contract:preinst-refuses-empty-root — apk-tools 25.12 completes the transaction
  (package registered, service started, exit 1).

That last item is the one I care about most, because it is a contradiction with this repo's own
design: since #902 the package is supposed to set no credential and refuse to install while root
has no password, yet the lane observes apk-tools completing the transaction anyway. Keeping that
asserted on purpose is exactly right — it is how the discrepancy stays visible instead of being
retitled as "expected". Someone should decide whether the fix belongs in the preinst, in the
packaging, or in an apk-tools workaround, but that decision needs the red to keep firing.

Terminology/coverage note (not a defect): the rc3 contract section correctly describes the
current design — "the package sets no credential at all" and "the preinst refuses any install
while root has no password" — and notes the earlier #894 staged-secret channel was absorbed by #897
and then removed by #902's stronger design, with "the history lives in #888". Recording the retired
mechanism rather than deleting the history is good practice for a lane that people will re-read.

Not re-run end-to-end, and why. Reproducing the lane needs a KVM host, a pristine OpenWrt image,
a built .apk and the documented operator env; the evidence in-tree is from the vlab lane
(ai-legion, x86-64). I verified the lane's own selftest (18/18) and the consistency of its
evidence file. I did not re-execute the 43-check run, so the pass:31/fail:12 figures are the
author's recorded measurement, not re-derived by me — stated plainly rather than implied.

Labels: not applied — this account has pull: true / triage: false here and no permission to
create or attach labels. Intended label for whoever can: medium (repeatable installer contract
lane, deliberately red; contains an assertion that contradicts the documented preinst design and
merits a decision).

FULL_REVIEW: ~/reports/reviews/tollgate-module-basic-go-pr892-kimi.md

[Amperstrand]

Local CI gate — PR #892: test(installer): contract lane for built packages on a pristine apk-era VM — #888/#890 acceptance, shipped red

Run of scripts/pr-verify.sh (the local CI gate: GitHub Actions is
org-blocked and the ngit lane fires only on maintainer mirror pushes,
so a PR ref gets no CI runs — this comment is the offline verification
evidence, produced on the pinned head SHA).

• Head: test/installer-contract-lane @ f0dfd4d1e5
• Base: main @ 27baf22fcb
• Gate: pr-verify 5a703563168f on ai-legion, started 2026-10-18 18:03 UTC, wall 438s
• Logs: /tmp/tollgate-pr-verify/892-20261018T175540Z (on the verifying host)

Leg	Result	Detail
hygiene	PASS	no conflict markers in the tracked tree (0s)
contract-fences	PASS	12 checks green; 1 skipped (absent from tree: entry-UI marker path fence) (4s)
packaging-sequential	PASS	13 suites (7s)
shell-suites	FAIL	red (1/16): uci-defaults-entry-ui-mapping_test.sh — logs in /tmp/tollgate-pr-verify/892-20261018T175540Z/shell-suites (45s)
packaging-concurrent	PASS	13 suites, all at once (3s)
go-battery	PASS	16 modules, gofmt/vet/build/-race (379s)

Verdict: 1 leg(s) RED — 5 green.

What this gate does not run (no vacuous green)

• spec-quote drift check (greatspectations tool + pinned nuts fetch)
• reproducibility / conformance lanes (make release-check territory)
• the ngit trigger/release-pipeline suites (release-check legs, not the
  per-PR lane; verified on the verifying host: release-pipeline 48/48
  green, trigger suite environment-red — this seat's nak v0.16.2 does
  not accept nsec-string args to key public, identical on main)
• router-visible behaviour (PRTA / bench — unit green ≠ router works)

failing-leg log tails

shell-suites — tail

--- uci-defaults-entry-ui-mapping_test.sh.log
== F. the marker is honoured on a same-version reinstall too
ok   reinstall: the driver exits 0
ok   reinstall (board): uhttpd.main (LuCI) is on the secondary :8090 pair
ok   reinstall (board): uhttpd.main (LuCI) is on the secondary :8443 pair
ok   reinstall (board): uhttpd.main holds NO entry-pair listener
ok   reinstall (board): uhttpd.main's home is /www
ok   reinstall (board): uhttpd.admin (the board) is on the entry :8080 pair
ok   reinstall (board): uhttpd.admin (the board) is on the entry :443 pair
ok   reinstall (board): the entry board keeps its webroot and error_page
ok   reinstall (board): exactly one section binds :8080 (admin)
ok   reinstall (board): exactly one section binds :443 (admin)
ok   reinstall (board): exactly one section binds :8090 (main)
ok   reinstall (board): exactly one section binds :8443 (main)

tests: 126 passed, 9 failed

[Amperstrand]

Evidence review — lane script verified; the methodology is the review-worthy half

run.sh parses; output idiom matches happy-path (ICCHECK/ICRESULT/ICEXIT). What earns it: pristine verification, not trust — the lane discovered firstboot -y && reboot silently keeps /etc on this image and forces reboot -f + asserts the clean state, because a false pristine makes credential assertions lie. Serial-bootstrapping a fresh VM when ssh is down is the exact manual plumbing from the 2026-10-11 vlab validation, now built in. End-to-end verified on the vlab lane per the body (rc1 apk, sha-pinned). This lane is the concrete backer of #925's release-check wiring (reviewed earlier this sweep) — merge-order-friendly in either direction, but #925's SKIP-without-artifact shape means this lane can land first and light up the gate leg. Mechanically ready; human gate.

[c03rad0r]

Technical review — requesting changes. Head f0dfd4d. The lane is valuable; shellcheck must pass.

The lane does what hand-validation did, and the two design decisions are right: pristine is proven, not assumed (firstboot -y silently kept /etc on this image, so the lane forces reboot -f and asserts the clean state), and the two known-reds are the acceptance criteria of #888/#890 rather than hidden. Shipping red with documented flip-to-green conditions is the correct shape for an acceptance lane.

Blocking: shellcheck -S warning on the new scripts. The repo's pre-commit hook runs it, so this tree cannot be committed cleanly as-is:

run.sh:80:1    SC2034  REPO_ROOT appears unused
run.sh:159:1   SC2120  finish references arguments, but none are ever passed
run.sh:212:5   SC2034  i appears unused
checks.sh:110:11 SC2155 Declare and assign separately to avoid masking return values

(There may be more; run the full shellcheck -S warning tests/installer-contract/*.sh.) These are small: drop the dead symbols, either pass the args finish expects or drop its parameters, and split the local x=$(...) in checks.sh.

Also confirm the evidence logs under tests/installer-contract/evidence/ contain no lab credentials/MACs (they are committed evidence; the repo's cred-gate should be run on them before merge).

The lane itself: land after shellcheck. Disposition: request-changes.

[c03rad0r]

Re-review — shellcheck clean. Approving.

The four warnings are fixed without changing the lane: local shape split from its assignment (SC2155), the unused REPO_ROOT/i removed, and finish no longer references arguments ("$*" and the unused extra unpack dropped). shellcheck -S warning on both scripts is clean; bash -n clean.

The lane itself (pristine-by-proof, two known-reds as the #888/#890 acceptance criteria) is unchanged and was reviewed as the right shape.

Disposition: land.