You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Payment consumed with no session when gate-open fails after successful Receive (no refund path) #403
Fund-safety edge (same family as #375): a successful payment is consumed even when opening the gate subsequently fails — the customer's ecash is swapped into the operator wallet, no session is granted, and there is no refund path.
Code path (verified on main @ 373770a):PurchaseSession (src/merchant/merchant.go:543–560) calls Receive (proofs swapped — irreversible once the mint accepts), then grantSessionAccess → openGateForSession → valve.OpenGateUntil → ndsctl auth. On failure, restoreSession (merchant.go:1065) rewinds only the in-memory customerSessions map — the customer's value stays in the operator wallet. The caller receives a kind-21023 session-error notice; nothing returns or credits the paid amount.
Live evidence (QEMU OpenWrt 24.10 lab, 2026-09-17, testnut):
12:03:44 PurchaseSession: calling Receive for mint=https://testnut.cashu.exchange token_amount=4 mac=02:11:22:33:44:51
12:03:46 PurchaseSession: Receive completed, amount=3, err=<nil> # 4 sat − 1 swap fee, now in operator wallet
12:03:46 Opening gate until timestamp duration_seconds=90 mac_address=02:11:22:33:44:51
12:03:47 Error authorizing MAC address error="exit status 1" # ndsctl auth failed
HTTP response: 400 kind-21023, code=session-error, "failed to open gate: error authorizing MAC: exit status 1". Reproduced twice (a second occurrence during soak bring-up at 12:06:30Z). Wallet balance grew by the received amount both times.
Trigger surface: any gate-open failure after a successful Receive. Observed triggers: (a) client MAC not yet registered in NDS at payment time (ndsctl auth exits 1 for unknown MACs), (b) NDS 5.0.2's ndsctl auth on an already-Authenticated client also exits 1 — first payment after fresh daemon state with a still-authed client hits this (noted on OpenTollGate/physical-router-test-automation#110).
Impact: worthless on signet/fakewallet, but on mainnet a paying customer whose gate-open fails loses the payment with no session and no recourse.
Suggested fix shapes (maintainer decision needed — not unilaterally chosen):
Refund path: on gate-open failure after successful Receive, return the received amount as a Cashu token in the notice event (drain-style), so the customer keeps custody.
Late grant: treat gate-open failure as non-fatal — the allotment is already recorded; retry ndsctl auth asynchronously (the valve already has retry machinery) and grant when NDS accepts.
Pre-flight: verify the client is NDS-registerable before calling Receive.
Why an issue, not a PR: the correct behavior is a product decision (refund semantics vs eventual-delivery), and it interacts with the NDS auth edges above. Happy to implement whichever shape is chosen — the lab that reproduced this can regression-test a fix within the hour.
Follow-up: code-level findings that narrow the fix, plus a severity call for v0.6.0.
Read the involved paths on current main again with the fix shapes in mind. Three findings change the picture:
1. A read-only pre-flight primitive already exists.valve.GetClientStats (src/valve/valve.go:338) wraps ndsctl json <mac> and already distinguishes "client not found" (empty {}, valve.go:356–360) from a live client, returning the NDS State field (valve.go:327). That answers both observed triggers before Receive is called, without authorizing anything:
Trigger (a) unknown MAC → probe returns "client not found" → reject with an actionable error while the customer still holds their token.
Trigger (b) NDS 5.0.2 exit-1 on already-Authenticated → probe says State == Authenticated → skip the redundant ndsctl auth; the gate is already open for that client. Pure valve/merchant change, no protocol work.
2. Bug-within-the-bug: the paid allotment is discarded too. On gate-open failure, restoreSession (src/merchant/merchant.go:1065) rewinds customerSessions, so the time/bytes the customer paid for is thrown away along with the session record — nothing is retained to grant later. Any "late grant" shape starts by not rewinding here.
3. The merchant side has no compensation counterpart — and can't reuse the upstream one. The upstream side (this router buying from an upstream) already has a compensation habit: recoverFailedPaymentToken (src/upstream_session_manager/token_recovery.go:16) claws the token back into the wallet or persists it to /etc/tollgate/tokens-to-recover.txt. That mechanism is structurally unavailable here: after a successful Receive the proofs are spent at the mint, so file-based token recovery cannot restore the customer — only a refund (mint a new token) or eventual delivery can. Also note nothing on this merchant failure path writes any persistent record tying the received proofs to the failed MAC; the only traces are log lines. Post-hoc reconciliation is log forensics.
Why not open-the-gate-first: auth is reversible (ndsctl deauth), settlement is not (the swap consumes the proofs). Ordering the reversible effect before the irreversible one buys a free-internet window bounded only by the mint roundtrip (~2 s in the repro), which is scriptable against MAC randomization, and turns every slow mint into auth/deauth flapping against NDS. Reversible-second is the safe ordering for the operator — it just needs a compensation path for the customer.
Refined fix shape — layered, each independently shippable:
L1 — read-only pre-flight before Receive (small): NDS presence + auth-state check via the existing ndsctl json wrapper; unknown MAC → actionable rejection, already-Authenticated → skip redundant auth. Kills both repro'd triggers. Zero abuse surface: it only refuses money or skips a no-op.
L2 — stop discarding the allotment: on gate-open failure keep the session record, retry ndsctl auth with backoff up to a deadline (valve already has the bounded retry machinery, valve.go:29-33; this path wants a longer, deadline-bounded variant), and adjust the session clock so the retry delay doesn't eat paid time.
L3 — refund backstop: if the deadline expires, mint a token for the gross pre-swap amount (operator eats the swap fee — the failure was operator-side) and return it inside the kind-21023 notice; delivery is reliable because the client is mid-HTTP-transaction on the portal at that moment. Refund-minting failure must be loud and persistent (file + error-level log), never best-effort, or this is the same bug one layer down.
L1 alone is small enough to land and regression-test in the lab quickly; L2/L3 cover the residual (check→auth TOCTOU, NDS fully down).
Severity for v0.6.0 (#339): by the project's own rubric — docs/tester-intake.md §4.2, "S1 — any wallet or funds symptom. This is stop-ship" — this qualifies: customer ecash consumed, no session, no persistent record on the merchant path. v0.6.0 is being cut specifically as a payment-fixes release; shipping it with a known consume-funds-without-service path in the primary purchase flow contradicts both the rubric and the release's theme. Suggested scope: L1 as a 0.6.0 requirement, L2/L3 either in 0.6.0 or as a disclosed fast-follow — maintainer call. The lab that reproduced this can regression-test L1 within the hour, same offer as before.
The pre-flight shape from the previous comment is now code on fix/403-payment-preflight:
valve.CheckClientState(mac) — read-only probe over ndsctl json <mac>; distinguishes definitively not registered (empty {}) from registered (+Authenticated), probe failures surface as errors so callers can fail open. Zero side effects — nothing is authorized.
PurchaseSession pre-Receive refusal — re-probes registration with the valve auth-retry cadence (5×400 ms) so the two-router reseller flow's asynchronous NDS registration is not refused on first sight; still unregistered → kind-21023 client-not-registered with a reconnect instruction, token untouched. Probe errors fail open — a broken probe must not become a payment DoS. This also covers the HTTP handler's fallback MAC 00:00:00:00:00:00, which can never be authorized.
authorizeMAC already-Authenticated short-circuit — on a failed auth attempt, probes once; Authenticated state → gate treated as open (trigger (b) defused at every call site, including the delayed-auth paths). Happy path unchanged: zero extra ndsctl calls when auth succeeds first try.
Verified: gofmt/vet/build clean across root+merchant+valve; go test -race -count=1 -tags testenv ./... green in all three; new tests pin refusal-before-Receive, fail-open, retry-until-registered, and the already-authed short-circuit. Unit-level only — the QEMU lab should regression-test both triggers on this branch.
Scope note: this is deliberately L1 only. The residual window (client vanishes between probe and auth, fully-dead NDS) still needs the L2 (keep the allotment, retry to a deadline) / L3 (refund-in-notice, gross amount) product decision — the issue stays open for that. For 0.6.0 scoping per #339: L1 is the S1-mitigating minimum; whether L2/L3 must also make the release is the remaining maintainer call.
Fresh reproduction from the clientd scenario battery (tests/cloud-lab/run-clientd-scenarios.sh on feature/tollgate-clientd): with ndsctl unavailable in the container, payments fail with payment rejected (HTTP 400): failed to open gate: exec: "ndsctl": executable file not found in $PATHafterPurchaseSession → Receive → swap already succeeded — the router log shows Receive completed, amount=1, err=<nil> / Amount after swap: 1 immediately before the 400. Token consumed, no session, no refund — exactly this issue's class. Suggest the refund/precheck path cover late gate-open failures as well, not just swap failures.
Closing-state note (2026-09-20). This issue spawned a body of work; here is where each part stands so the remainder is a deliberate choice, not a gap:
L1 (defuse the known triggers) — done and validated twice:fix(merchant): refuse payment when NDS has no client session for the MAC (#403 L1) #412 refuses payment for NDS-unregistered MACs before Receive (read-only probe, fail-open) and short-circuits the already-Authenticated exit-1 case. Unit-pinned, cloud-lab green, and QEMU-router-validated on real nodogsplash 5.0.2 (both triggers, token provably unspent, 1.6s refusal). Awaiting felix review, MERGEABLE.
L2 (stop discarding the allotment; retry to a deadline) and L3 (refund-in-notice) — still open by design: they change session semantics and the 21023 notice schema (wire contract) and are a maintainer product decision. The residual window (client vanishes between probe and auth; NDS fully dead) remains until they land. Suggested owner call: scope decision at v0.6.x planning.
Fresh measurement for the decision record (2026-10-05, #535 lane, current main post-#558/#595).
The 2026-09-26 review of #535 flagged that this window's aftermath "may legitimately have changed" under #558 (Receive-outcome recording) and #595 (loud unappliable grant). Measured: it has not changed. The kill scenario's aftermath, re-run on rebased main: the token's proofs are spent at the mint, the retry is refused (21023), no session was ever granted, and no refund path exists — customer value consumed with neither service nor refund, exactly as when the window was first recorded. The good news in the same row: no-output-reuse=pass — the counter discipline holds across the kill/restart (fresh derivation, no re-exposure), and restart-converges=pass.
This is now the only red invariant on the kill scenario, and one of two overall (with swap-timeout's service-or-refund, same root). The reconciliation building blocks that exist today: receiveReference (the customer-quotable handle already handed out on the outcome-unknown path), recordLateReceiveOutcome (log-only late-outcome recorder), and the quote_store.go persistence precedent. The open decision — refund vs late-grant, per the AGENTS.md rule "decide refund vs late-grant explicitly; do not silently drop it" — still needs the operator's call before an ADR can be drafted.
Activity
Amperstrand commentedon Sep 19, 2026
Follow-up: code-level findings that narrow the fix, plus a severity call for v0.6.0.
Read the involved paths on current
mainagain with the fix shapes in mind. Three findings change the picture:1. A read-only pre-flight primitive already exists.
valve.GetClientStats(src/valve/valve.go:338) wrapsndsctl json <mac>and already distinguishes "client not found" (empty{}, valve.go:356–360) from a live client, returning the NDSStatefield (valve.go:327). That answers both observed triggers beforeReceiveis called, without authorizing anything:State == Authenticated→ skip the redundantndsctl auth; the gate is already open for that client. Pure valve/merchant change, no protocol work.2. Bug-within-the-bug: the paid allotment is discarded too. On gate-open failure,
restoreSession(src/merchant/merchant.go:1065) rewindscustomerSessions, so the time/bytes the customer paid for is thrown away along with the session record — nothing is retained to grant later. Any "late grant" shape starts by not rewinding here.3. The merchant side has no compensation counterpart — and can't reuse the upstream one. The upstream side (this router buying from an upstream) already has a compensation habit:
recoverFailedPaymentToken(src/upstream_session_manager/token_recovery.go:16) claws the token back into the wallet or persists it to/etc/tollgate/tokens-to-recover.txt. That mechanism is structurally unavailable here: after a successfulReceivethe proofs are spent at the mint, so file-based token recovery cannot restore the customer — only a refund (mint a new token) or eventual delivery can. Also note nothing on this merchant failure path writes any persistent record tying the received proofs to the failed MAC; the only traces are log lines. Post-hoc reconciliation is log forensics.Why not open-the-gate-first: auth is reversible (
ndsctl deauth), settlement is not (the swap consumes the proofs). Ordering the reversible effect before the irreversible one buys a free-internet window bounded only by the mint roundtrip (~2 s in the repro), which is scriptable against MAC randomization, and turns every slow mint into auth/deauth flapping against NDS. Reversible-second is the safe ordering for the operator — it just needs a compensation path for the customer.Refined fix shape — layered, each independently shippable:
Receive(small): NDS presence + auth-state check via the existingndsctl jsonwrapper; unknown MAC → actionable rejection, already-Authenticated → skip redundant auth. Kills both repro'd triggers. Zero abuse surface: it only refuses money or skips a no-op.ndsctl authwith backoff up to a deadline (valve already has the bounded retry machinery,valve.go:29-33; this path wants a longer, deadline-bounded variant), and adjust the session clock so the retry delay doesn't eat paid time.L1 alone is small enough to land and regression-test in the lab quickly; L2/L3 cover the residual (check→auth TOCTOU, NDS fully down).
Severity for v0.6.0 (#339): by the project's own rubric — docs/tester-intake.md §4.2, "S1 — any wallet or funds symptom. This is stop-ship" — this qualifies: customer ecash consumed, no session, no persistent record on the merchant path. v0.6.0 is being cut specifically as a payment-fixes release; shipping it with a known consume-funds-without-service path in the primary purchase flow contradicts both the rubric and the release's theme. Suggested scope: L1 as a 0.6.0 requirement, L2/L3 either in 0.6.0 or as a disclosed fast-follow — maintainer call. The lab that reproduced this can regression-test L1 within the hour, same offer as before.
Amperstrand commentedon Sep 19, 2026
L1 implemented — PR #412.
The pre-flight shape from the previous comment is now code on
fix/403-payment-preflight:valve.CheckClientState(mac)— read-only probe overndsctl json <mac>; distinguishes definitively not registered (empty{}) from registered (+Authenticated), probe failures surface as errors so callers can fail open. Zero side effects — nothing is authorized.PurchaseSessionpre-Receiverefusal — re-probes registration with the valve auth-retry cadence (5×400 ms) so the two-router reseller flow's asynchronous NDS registration is not refused on first sight; still unregistered → kind-21023client-not-registeredwith a reconnect instruction, token untouched. Probe errors fail open — a broken probe must not become a payment DoS. This also covers the HTTP handler's fallback MAC00:00:00:00:00:00, which can never be authorized.authorizeMACalready-Authenticated short-circuit — on a failed auth attempt, probes once;Authenticatedstate → gate treated as open (trigger (b) defused at every call site, including the delayed-auth paths). Happy path unchanged: zero extra ndsctl calls when auth succeeds first try.Verified:
gofmt/vet/buildclean across root+merchant+valve;go test -race -count=1 -tags testenv ./...green in all three; new tests pin refusal-before-Receive, fail-open, retry-until-registered, and the already-authed short-circuit. Unit-level only — the QEMU lab should regression-test both triggers on this branch.Scope note: this is deliberately L1 only. The residual window (client vanishes between probe and auth, fully-dead NDS) still needs the L2 (keep the allotment, retry to a deadline) / L3 (refund-in-notice, gross amount) product decision — the issue stays open for that. For 0.6.0 scoping per #339: L1 is the S1-mitigating minimum; whether L2/L3 must also make the release is the remaining maintainer call.
fix(merchant): refuse payment when NDS has no client session for the MAC
Amperstrand commentedon Sep 20, 2026
Fresh reproduction from the clientd scenario battery (tests/cloud-lab/run-clientd-scenarios.sh on feature/tollgate-clientd): with ndsctl unavailable in the container, payments fail with
payment rejected (HTTP 400): failed to open gate: exec: "ndsctl": executable file not found in $PATHafterPurchaseSession → Receive → swapalready succeeded — the router log showsReceive completed, amount=1, err=<nil>/Amount after swap: 1immediately before the 400. Token consumed, no session, no refund — exactly this issue's class. Suggest the refund/precheck path cover late gate-open failures as well, not just swap failures.Amperstrand commentedon Sep 20, 2026
Closing-state note (2026-09-20). This issue spawned a body of work; here is where each part stands so the remainder is a deliberate choice, not a gap:
Receive(read-only probe, fail-open) and short-circuits the already-Authenticated exit-1 case. Unit-pinned, cloud-lab green, and QEMU-router-validated on real nodogsplash 5.0.2 (both triggers, token provably unspent, 1.6s refusal). Awaiting felix review, MERGEABLE.Nothing in this issue blocks the v0.6.0 tag once #412 merges; L2/L3 are the tracked remainder.
fix(merchant): refuse payment when NDS has no client session for the …
docs(agents): fund-safety, crash-consistency and distributed-transact…
test(cloud-lab): conformance fast-subset lane over the PRTA fault proxy
fix(session): renew after expiry; add an explicit session state (none…
test(cloud-lab): conformance fast-subset lane over the PRTA fault proxy
test(merchant): the payment pre-flight's test seam is per merchant (f…
test(cloud-lab): conformance fast-subset lane over the PRTA fault proxy
Amperstrand commentedon Oct 5, 2026
Fresh measurement for the decision record (2026-10-05, #535 lane, current main post-#558/#595).
The 2026-09-26 review of #535 flagged that this window's aftermath "may legitimately have changed" under #558 (Receive-outcome recording) and #595 (loud unappliable grant). Measured: it has not changed. The kill scenario's aftermath, re-run on rebased main: the token's proofs are spent at the mint, the retry is refused (
21023), no session was ever granted, and no refund path exists — customer value consumed with neither service nor refund, exactly as when the window was first recorded. The good news in the same row:no-output-reuse=pass— the counter discipline holds across the kill/restart (fresh derivation, no re-exposure), andrestart-converges=pass.This is now the only red invariant on the kill scenario, and one of two overall (with swap-timeout's
service-or-refund, same root). The reconciliation building blocks that exist today:receiveReference(the customer-quotable handle already handed out on the outcome-unknown path),recordLateReceiveOutcome(log-only late-outcome recorder), and thequote_store.gopersistence precedent. The open decision — refund vs late-grant, per the AGENTS.md rule "decide refund vs late-grant explicitly; do not silently drop it" — still needs the operator's call before an ADR can be drafted.test(cloud-lab): conformance fast-subset lane over the PRTA fault proxy
fix(merchant): a paid purchase whose gate cannot open is an owed enti…
fix(merchant): a paid purchase whose gate cannot open is an owed enti…
fix(merchant): a paid purchase whose gate cannot open is an owed enti…
test(cloud-lab): conformance fast-subset lane over the PRTA fault proxy
test(cloud-lab): conformance fast-subset lane over the PRTA fault pro…
620: merge main forward — resolve the CHANGELOG conflict (keep both s…
fix(merchant): the boot receive-intent reconcile sequences after the …
fix(merchant): the boot receive-intent reconcile sequences after the …