Repository navigation
test(packaging): rootfs-container tier for uci-defaults validation - #530
Amperstrand wants to merge 3 commits into
Conversation
|
Verification run (worktree at head The tier reproduces its claims exactly.
What the harness genuinely buys. The containers run real BusyBox ash + the real But the tier is blind to the #516/#518 class — verified, not assumed. The driver's assertion surface (
So the one documented real-uci bug class in this area — the exact motivation for #521's proposal — passes through this tier today. The fix is nearly free given the existing rerun loop: in the driver, after each setup pass, assert the allow list is stable, e.g. Smaller notes: the CHANGELOG entry (line 9) is in the right Disposition: land-with-followups — (1) add the nodogsplash allow-list stability assertion to the driver (the gap proven above; this is the tier's headline raison d'être), (2) add the |
|
Wave-3 follow-up pushed ( |
|
Technical review (evidence pass — this one I could run whole). What it does. Adds the rootfs-container tier for uci-defaults validation (#521): the real first-boot setup script executed end-to-end inside real OpenWrt userspace — BusyBox ash, the real Verification — the full tier, executed on this host: Disposition: land — strongest-tier validation for exactly the first-boot logic the release ships; only blocker is the routine rebase over the overnight wave. |
e3ce73c to
c0c8555
Compare
|
CHANGES REQUESTED — reviewed at pinned head [BLOCK] The branch no longer contains this PR. Live head On the rootfs-tier content itself (identical at [RISK] tests/uci-defaults-rootfs_test.sh — binding assertions silently skip when the AP section is missing. In the in-container driver, [NIT] assert_allowlist asserts six ports; the writer writes seven. The driver's port loop (≈line 397) covers 2121/8080/2050/2051/8090/8443, but the setup script (lines 424–426) also writes Verification (mine, at Restore the branch, close the RISK, and this lands — the tier's design is right, the fixtures are honest (section names deliberately disagree with bands), and it runs clean end-to-end against real |
felixfelix-bot
left a comment
There was a problem hiding this comment.
CHANGES REQUESTED — reviewed at pinned head 3456de9cca24c0b52e4abb4f0ae8b6f19c5c1a60 (rootfs tier, now dangling) and live head c0c8555a4c868ba4803186924f68922923850ed2; the two are different PRs.
[BLOCK] The branch no longer contains this PR. Live head c0c8555a4c86 (force-pushed 2026-09-24 12:23Z/12:26Z) carries the wallet-recover feature: the seven live commits (d1e97175…c0c8555a) are an exact SHA-prefix of open PR #549's eight (head f5fc6f14) — commit identity, not just a similar diff. The live diff is 22 files, 20 of them under src/ (tollwallet, cli, merchant) plus CHANGELOG.md and docs/operator-guide.md, with tests/uci-defaults-rootfs_test.sh absent entirely, while the title and body still describe the rootfs tier (the live CHANGELOG entry now advertises #549's wallet-recover). Merging #530 as-is merges wallet-recover under the wrong title and drops the rootfs tier entirely. Restore the rootfs head — e3ce73c23183f29273d77a7e88674990b1076b09 (final rootfs head after rebase; test file identical, blob 8df492df) or 3456de9cca24 — and let wallet-recover land via #549. (GitHub still serves both dangling SHAs; I fetched and verified them.)
On the rootfs-tier content itself (identical at 3456de9c and e3ce73c2; executed on a docker host at 3456de9c):
[RISK] tests/uci-defaults-rootfs_test.sh — binding assertions silently skip when the AP section is missing. In the in-container driver, assert_bindings guards each check with [ -n "$S2G" ] && check … (heredoc, ≈lines 373–376 of the test): if setup stops creating the AP sections entirely, the assertion is skipped, not failed. Demonstrated by neutering the four setup_band_ap call sites in packaging/files/etc/uci-defaults/99-tollgate-setup at 3456de9c and running the 24.10 image: 209 passed / 4 failed, exit 1 — all four failures from upgrade-misbound, the one topology whose fixture carries pre-existing tollgate_* sections; the other six topologies stayed green while emitting zero binding assertions. A fully-neutered setup on six of seven topologies still reports a green tier. The AP-absence class is exactly #103/#173 ("APs missing after reinstall"). Fix: when a band's radio exists, a missing tollgate_2g_open/tollgate_5g_open section must FAIL, not skip.
[NIT] assert_allowlist asserts six ports; the writer writes seven. The driver's port loop (≈line 397) covers 2121/8080/2050/2051/8090/8443, but the setup script (lines 424–426) also writes allow tcp port 443 — the operator-lockout entry this whole contract exists to guard. Verified in-container: the post-setup users_to_router list carries 7 entries; the tier asserts 6. The offline uci-defaults-same-version-allowlist_test.sh does assert :443 (its ALL_PORTS, line 153), so this is a completeness gap in the new tier rather than an uncovered contract — but the tier's own header (lines 388–389) codifies "six anchored entries", which is what makes the gap look deliberate. Adding 443 to the loop — and to the comment — is two tokens.
Verification (mine, at 3456de9c): full tier on both images: 470 passed, 0 failed, exit 0. Offline siblings: band 22/0, nodogsplash-443 19/0, private-subnet 12/0, same-version-allowlist 43/0 — all exit 0. shellcheck -S warning clean. gitleaks on the diff: clean. No CI runs exist for the branch (repo-wide Actions wedge, not this PR's doing).
Restore the branch, close the RISK, and this lands — the tier's design is right, the fixtures are honest (section names deliberately disagree with bands), and it runs clean end-to-end against real uci on both package eras.
|
Technical review (delta pass since the 2026-09-24 changes-requested) — PR head The blocker stands, and it got worse. The branch still does not contain this PR: the live head carries the seven-commit wallet-recover series ( New since the last review: #549 was rewritten, orphaning the carried series. At the last review this head's commits were an exact SHA-prefix of #549's then-eight; they no longer are — none of the seven SHAs here is an ancestor of #549's current head The carried content is real and green — which is exactly why it must land via #549, not here. I ran the touched modules at this head: Action (unchanged, now more urgent):
Disposition: request-changes — sole blocker unchanged: put the PR back on its branch. The tier itself was fully verified twice in this thread and lands the moment the branch is whole again. |
e3ce73c to
248aae6
Compare
|
Branch restored and rebased (post-review maintenance). This branch — like #557, #569 and #570 — had been force-pushed over with #549's wallet-recover series; the rootfs tier survived only as dangling commits. Restored from Remaining pre-merge items from the review are unchanged and still apply to the restored content: the binding assertions must fail, not skip, when a band's radio exists but its |
|
CHANGES REQUESTED — reviewed at live head Good news first: the branch is whole again — the tier is back, both commits apply cleanly, and the prior blocker is gone. But the rebase onto current main imported two main-side contract changes, and the test file was not updated to match. I ran the tier end-to-end on a docker host at this head, both images: Two failure classes, exactly, on both images — everything else (all band bindings across the seven topologies, private-AP bindings and credential preservation, STA preservation, phantom-AP absence on single-band, the 2121/2050/2051 entries and their exactly-once counts) passes:
One observation worth its own line: Verified clean at this head: |
felixfelix-bot
left a comment
There was a problem hiding this comment.
CHANGES REQUESTED at head 248aae6 — see the review comment for details and evidence: the tier is empirically red at its own head (288 passed / 182 failed, exit 1, executed on a docker host at this head) from two rebase-imported contract drifts: (1) the allow-list loop at tests/uci-defaults-rootfs_test.sh:397 asserts presence of 8080/8090/8443, which the writer on main now actively removes (99-tollgate-setup:1449-1456) — mirror the offline sibling's JOURNEY/ADMIN split; (2) the rerun probe at :433 greps 'Flag matches', deleted by #578 — use the boundary anchor 'Setup branch VERIFY([[:space:]]|$)' like the three sibling suites; plus (3) CHANGELOG.md:1184 ships '134 assertions' — the real count is ~470; and (4, carried) the binding assertions at :373-374 skip instead of fail when the AP section is missing. Design is right; all fixes are confined to the driver heredoc and the CHANGELOG figure.
The offline uci-defaults tests drive the setup script's functions through a fake uci: fast, but blind to real-uci semantics and GNU-vs-BusyBox differences — a fake-uci matrix once passed what real uci then failed on private-network setup. Add tests/uci-defaults-rootfs_test.sh: it runs the real script end-to-end inside openwrt/rootfs containers (BusyBox ash, the real uci binary) against seven fixture wireless topologies with authored band ground truth, on both the 24.10 (opkg) and 25.12-snapshot (apk) images, including the same-version verify path. The rootfs image ships without /etc/config defaults, so the driver seeds minimal ones — which incidentally exercises the script's designed no-LAN guard. Skips with exit 0 where no docker exists, so it can ride runners without container support (#521).
…tract The wave-3 review's follow-up: the tier ran 134/0 yet a planted #516-class regression still passed — zero nodogsplash allow-list assertions, the exact class the tier exists to catch. assert_allowlist() now runs after both setup paths (full setup and the same-version rerun — the rerun being the duplicate-class witness): for each of the six contracted entries (2121, 8080, 2050, 2051, 8090, 8443) it asserts presence via the same anchored whole-field match as the writer's uci_list_has_port (:8443 cannot satisfy a :443-class check) and asserts the entry appears exactly once (occurrence-counted with grep -o | wc -l — grep -c counts lines, which is always 1 on a single-line list and would never see a duplicate). Also hardens a pre-existing SC2164 shellcheck warning in the same file (cd without || exit). Verified on real OpenWrt userspace (openwrt/rootfs:x86-64-openwrt-24.10, 7 topologies): 235 passed, 0 failed — up from 134, +12 assertions per topology (6 presence + 6 exactly-once) after each of the two setup paths. Synthetic dup/trip/absent probes: counts 2/3/0 as expected.
… narrowed allow-list contract Review response (2026-09-30 changes-requested at 248aae6, plus the two standing items from the 2026-09-26/27 passes): assert_bindings(): a band whose radio exists but whose tollgate_*_open section is missing now FAILs instead of silently skipping — the #103/#173 reinstall class. Previously a fully-neutered AP setup stayed green on 6 of 7 topologies. assert_allowlist(): mirrors the contract main now ships (and the offline sibling's JOURNEY_PORTS/ADMIN_PORTS split) instead of the retired one. Journey ports 2121/2050/2051 asserted present exactly once; admin surfaces 8080/443/8090/8443 asserted ABSENT from the pre-auth list — this supersedes the earlier "add 443" item: main's writer removes 443 (operator-lockout entry), so the tier asserts its absence. 443 is now asserted on every topology, both setup paths. Rerun probe: "Flag matches" no longer exists (#578 replaced the branch log with "Setup branch $BRANCH — $REASON"). The raw script this container runs is never packaged, so its recorded marker is unorderable and the same-version rerun legitimately lands on VERIFY_REPAIR, not plain VERIFY; the assertion accepts both verify-family tokens with a boundary after the token (never matching a hypothetical other verdict) and fails on FULL or a missing verdict line. CHANGELOG: the entry counted 134 assertions; the tier emits 414 — and it now names the binding and allow-list contracts it asserts. PR link (#530) added per house style. Verified on real OpenWrt userspace, both images (24.10 + 25.12-SNAPSHOT) x seven topologies: 414 passed, 0 failed, exit 0 (~20 s warm; skip path exit 0 without docker). Discrimination controls, one image: neutered AP creation fails all seven topologies (was green on 6 of 7); a writer that re-admits allow tcp port 443 fails all seven topologies. Offline siblings re-run green: band 22/0, same-version-allowlist 51/0, nodogsplash-convergence 29/0, private-subnet 12/0. bash -n and shellcheck -S warning clean. No Go files touched.
248aae6 to
3f254b9
Compare
|
Fix-and-rebase for the standing review items — new head Resolutions, per finding:
Verification (this run, at
The tier's CI wiring (#457) remains the known open follow-up — no workflow invokes this file yet. |
|
Rebased onto What conflicted: Nothing was rewritten but history: the replayed script is byte-identical to the pre-rebase head ( Verified on the rebased tree:
To move this PR's head (whoever has write on the org branch — the branch head lives in git fetch https://github.com/felixfelix-bot/tollgate-module-basic-go pr/rootfs-validation-tier-rebased
git checkout rootfs-validation-tier
git reset --hard FETCH_HEAD
git push --force-with-lease origin rootfs-validation-tierA force-push may dismiss the existing approval if the branch protection dismisses stale reviews — re-approving is the price of a clean rebase. If you would rather not wait on the author, say so and I will open the rebased version as a replacement PR against |
|
Rebased content offered as #671 (base git fetch https://github.com/felixfelix-bot/tollgate-module-basic-go pr/rootfs-validation-tier-rebased
git checkout rootfs-validation-tier
git reset --hard FETCH_HEAD
git push --force-with-lease origin rootfs-validation-tierBoth routes land the same single commit. I will close #671 myself if you prefer the force-push. |
…rebase of #530 (#671) * test(packaging): rootfs-container tier for uci-defaults validation The offline uci-defaults tests drive the setup script's functions through a fake uci: fast, but blind to real-uci semantics and GNU-vs-BusyBox differences — a fake-uci matrix once passed what real uci then failed on private-network setup. Add tests/uci-defaults-rootfs_test.sh: it runs the real script end-to-end inside openwrt/rootfs containers (BusyBox ash, the real uci binary) against seven fixture wireless topologies with authored band ground truth, on both the 24.10 (opkg) and 25.12-snapshot (apk) images, including the same-version verify path. The rootfs image ships without /etc/config defaults, so the driver seeds minimal ones — which incidentally exercises the script's designed no-LAN guard. Skips with exit 0 where no docker exists, so it can ride runners without container support (#521). * test(packaging): rootfs tier asserts the NDS allow-list stability contract The wave-3 review's follow-up: the tier ran 134/0 yet a planted #516-class regression still passed — zero nodogsplash allow-list assertions, the exact class the tier exists to catch. assert_allowlist() now runs after both setup paths (full setup and the same-version rerun — the rerun being the duplicate-class witness): for each of the six contracted entries (2121, 8080, 2050, 2051, 8090, 8443) it asserts presence via the same anchored whole-field match as the writer's uci_list_has_port (:8443 cannot satisfy a :443-class check) and asserts the entry appears exactly once (occurrence-counted with grep -o | wc -l — grep -c counts lines, which is always 1 on a single-line list and would never see a duplicate). Also hardens a pre-existing SC2164 shellcheck warning in the same file (cd without || exit). Verified on real OpenWrt userspace (openwrt/rootfs:x86-64-openwrt-24.10, 7 topologies): 235 passed, 0 failed — up from 134, +12 assertions per topology (6 presence + 6 exactly-once) after each of the two setup paths. Synthetic dup/trip/absent probes: counts 2/3/0 as expected. * test(packaging): rootfs tier fails on absent AP sections, asserts the narrowed allow-list contract Review response (2026-09-30 changes-requested at 248aae6, plus the two standing items from the 2026-09-26/27 passes): assert_bindings(): a band whose radio exists but whose tollgate_*_open section is missing now FAILs instead of silently skipping — the green on 6 of 7 topologies. assert_allowlist(): mirrors the contract main now ships (and the offline sibling's JOURNEY_PORTS/ADMIN_PORTS split) instead of the retired one. Journey ports 2121/2050/2051 asserted present exactly once; admin surfaces 8080/443/8090/8443 asserted ABSENT from the pre-auth list — this supersedes the earlier "add 443" item: main's writer removes 443 (operator-lockout entry), so the tier asserts its absence. 443 is now asserted on every topology, both setup paths. Rerun probe: "Flag matches" no longer exists (#578 replaced the branch log with "Setup branch $BRANCH — $REASON"). The raw script this container runs is never packaged, so its recorded marker is unorderable and the same-version rerun legitimately lands on VERIFY_REPAIR, not plain VERIFY; the assertion accepts both verify-family tokens with a boundary after the token (never matching a hypothetical other verdict) and fails on FULL or a missing verdict line. CHANGELOG: the entry counted 134 assertions; the tier emits 414 — and it now names the binding and allow-list contracts it asserts. PR link (#530) added per house style. Verified on real OpenWrt userspace, both images (24.10 + 25.12-SNAPSHOT) x seven topologies: 414 passed, 0 failed, exit 0 (~20 s warm; skip path exit 0 without docker). Discrimination controls, one image: neutered AP creation fails all seven topologies (was green on 6 of 7); a writer that re-admits allow tcp port 443 fails all seven topologies. Offline siblings re-run green: band 22/0, same-version-allowlist 51/0, nodogsplash-convergence 29/0, private-subnet 12/0. bash -n and shellcheck -S warning clean. No Go files touched. --------- Co-authored-by: Amperstrand <amperstrand@localhost>
|
Closing as landed... |
|
There is nothing left to resolve here: this PR's content is already in Proof against the current tree (
The Fresh evidence on today's Recommended action: close this PR as landed. (I'd close it myself, but |
|
Superseded by #671 — same rootfs-container tier for uci-defaults validation, rebased onto current main and merged there. Closing to keep the queue clean. |
Implements the runnable lane from #521 (CI wiring deliberately left as a maintainer choice — see the end).
What
tests/uci-defaults-rootfs_test.sh: runs the first-boot setup script end-to-end inside real OpenWrt userspace —openwrt/rootfscontainers with BusyBox ash and the realucibinary — against seven fixture/etc/config/wirelesstopologies with authored band ground truth (section names deliberately disagree with bands on the swapped/misbound fixtures, because radio numbering carries no band meaning). Both package ecosystems are covered:x86-64-openwrt-24.10(opkg) andx86-64-25.12-SNAPSHOT(apk), overridable viaUCI_ROOTFS_IMAGES.Per topology the in-container driver: seeds minimal router defaults (the rootfs image ships without
/etc/config/networketc. — this also exercises the script's designed no-LAN guard), installs the fixture, runs the full-setup path, asserts bindings/STA-preservation/credential-preservation/phantom-AP absence, then reruns same-version and asserts the verify-only path kept the bindings. Style matches the siblinguci-defaults-*_test.shfiles (single self-contained file, ok/bad counters); skips with exit 0 where no docker exists.Why this tier
The offline tests remain the fast loop, but a fake-uci matrix once passed a matrix that real uci then failed (private-network setup differed under real semantics). Real BusyBox/uci is where #459-class and toolchain-divergence bugs live.
Verification (this run)
main— including the rebased band logic from fix: assign radios by band and expose the band on the scan path (rebased #452 + band exposure) #490 and the fix(packaging,captive-portal), feat(setup): nodogsplash dependency, pre-auth :443, collision-aware private subnet #513 private-subnet changes.skip:+ exit 0.bash -nclean; sibling suite green (band 22, nodogsplash-443 19, private-subnet 12, same-version-allowlist 43 — all 0 fail).make go-battery: 16 modules green (no Go changes).CI wiring
Left out on purpose: the GHA twin is being archived (#457) and ngit-side container support is unconfirmed. The test is designed to ride any runner:
bash tests/uci-defaults-rootfs_test.shwith docker → runs; without → skips. Wherever containers land, this drops in unchanged.Changelog
Changed / Internalentry added.