You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
wallet: swap-timeout retry re-exposes derivation outputs (measured by the #535 conformance lane on current main) — the #257/#266/#480 brick class #640
The conformance fast-subset lane (#535), re-run on current main (post-#558/#595, branch head rebase 2026-10-05), measured the swap-timeout-retry scenario re-exposing deterministic derivation outputs after an ambiguous outcome — the exact wallet-brick class of #257/#266/#480, forbidden by the fund-safety rules in AGENTS.md ("Never reuse deterministic Cashu derivation outputs... Re-derivation triggers mint error 10002... and has repeatedly bricked wallets").
What the lane measured
The PRTA fault proxy (which observes every blinded message the daemon sends to the mint) recorded the daemon's swap-route derivation outputs sighted twice after the proxy processed the first swap and dropped the response (the ambiguous-outcome window): the same output set was re-sent and the retry succeeded — the payment returned 1022 anyway. A mint that enforces duplicate-output detection (error 10002) would have bricked that derivation range instead; the lab mint tolerates it, which is why payments keep working while the invariant is violated.
Evidence (from the lane run, .conformance/evidence/swap_timeout_retry.log + the proxy observation log): blinded-message digests with total: 2, phase_delta: 2 on swap routes; verdict row swap-timeout-retry / no-output-reuse=fail, restart-converges=pass.
The open question the lane cannot answer
Whether the re-send is gonuts-tollgate retry logic re-deriving from the same counter range, or Go's net/http transport replaying the request body after the connection was cut — the fix differs (fresh-counter regeneration on ambiguous outcome vs. disabling request replay / idempotency keys). Per the fund-safety rules, on timeout the wallet must query proof/quote state before regenerating (NUT-07 checkstate), and any retry must derive from a freshly incremented counter. The reference model is CDK's wallet saga (crates/cdk/src/wallet/swap/saga/) — the ambiguous-outcome reconciliation is exactly what its crash-safe state machine exists for.
Reproduce
cd tests/cloud-lab
./conformance/run-conformance.sh # needs docker + a PRTA sibling checkout
# row: swap-timeout-retry no-output-reuse=fail (evidence in .conformance/evidence/)
Filed per the #535 review's follow-up list ("file the swap-timeout re-exposure follow-up against current main, not the fork point") — the original observation predated #558/#595 and is now re-confirmed against them: neither changed the re-exposure behavior.
Root cause found (code-level, verified against gonuts-tollgate v0.12.1 as bumped into this module).
The re-send is the wallet client's own transport-error retry, not net/http replay and not swapWithRetry
Three candidates were checked:
Wallet.swapWithRetry (wallet/wallet.go:885): retried — it only fires on the mint's BlindedMessageAlreadySignedErrCode (10002), re-derives fresh blinded messages, and increments the keyset counter before sending. That path is disciplined (fix: bump gonuts-tollgate v0.7.1 → v0.7.4 for swap-counter race #266's fix); it is not the re-exposure.
Go net/http transport-level replay (GetBody + reused connection): ruled out empirically — a minimal repro (POST with a *bytes.Reader body over a pooled connection that the server kills while idle, same client shape as the fork's) shows the transport opens a fresh connection and sends the request once; a non-GET/HEAD without Idempotency-Key is not replayable in Go's transport.
httpPostWithRetry (wallet/client/client.go:459-474): this is it. On a transport error it sleeps a backoff and re-POSTs the identical serialized body:
A dropped response after the mint processed the swap (the lane's drop_response) surfaces as exactly such a transport error: the same derivation outputs are re-sent to the mint. The observed chain, matching every lane symptom: first swap processed by the mint → response dropped → transport error → same-body retry (the re-exposure, digests sighted twice) → mint answers 10002 for the already-signed outputs → swapWithRetry's legitimate fresh re-derivation completes the swap → payment returns 1022. The lab mint tolerates the sequence, which is why payments keep working while the invariant is broken; a mint with strict duplicate-output enforcement is the brick scenario the fund-safety rules describe.
Why this is the wrong retry, per the repo's own rules
AGENTS.md: "Ambiguous network results must be reconciled, never blindly retried. A swap/melt timeout does not mean failure — the mint may have processed it. On timeout, query proof/quote state before regenerating or retrying." A transport error on a money-moving POST is precisely an ambiguous outcome. The same-body retry is also applied to /v1/mint/bolt11, /v1/melt/quote/bolt11 and /v1/melt/bolt11 — melt retries are the same class (a melt whose response is dropped may have paid the invoice).
Fix direction (fork-level — OpenTollGate/gonuts-tollgate, then the three-go.mod bump)
The transport-error and 5xx retry must be per-route: never for the money-moving POSTs (/v1/swap, /v1/mint/bolt11, /v1/melt/quote/bolt11, /v1/melt/bolt11). For those, an ambiguous outcome returns an error that the caller reconciles — NUT-07 checkstate for the swap outputs' inputs, quote state for melts — before any regeneration (which must use a freshly incremented counter, per the existing 10002 path).
Retrying GETs (/v1/keys, /v1/checkstate, keyset endpoints) on transport errors is fine and should stay.
The 429 retry-with-Retry-After on POSTs is a separate question: a 429 means the mint did not process the request (it refused it), so a same-body retry there is safe in principle — but only if the mint's 429 is answered before processing, which the current single-status-check shape cannot distinguish for sure. Keep it, but document the assumption.
Repro: tests/cloud-lab/conformance/run-conformance.sh, row swap-timeout-retry / no-output-reuse=fail (evidence under .conformance/evidence/swap_timeout_retry.log + the fault-proxy observation log).
Fix state: root-caused, fixed, and lane-proven — awaiting review/merge chain.
Root cause (comment above): the fork's httpPostWithRetry re-POSTed the identical body on any transport error.
Fix: gonuts-tollgate#35 — state-changing POSTs are single-shot on no-answer (new AmbiguousOutcomeError, reconcile-first contract), 429 keeps the same-body retry, checkstate keeps its full retry.
End-to-end proof (posted on the PR): the module's conformance lane re-run with the fork branch pinned (v0.12.2-0.20261005164740-3c054dfc880d) — swap-timeout-retry / no-output-reuse flips fail → pass; each blinded-message digest is now sighted exactly once even when the swap response is dropped after the mint processed it.
Merge chain to actually close this: fork #35 → tag → the three-go.mod bump PR in this repo → re-run the lane on the bumped main. Two follow-throughs for the bump PR: teach isAmbiguousMintOutcomeError in src/merchant to match client.AmbiguousOutcomeError (so recordLateReceiveOutcome labels the no-answer case precisely), and expect the swap-timeout-retry row to remain fail on service-or-refund only — that is #403/#258, deliberately not masked here.
Chain closed through the tag: fork #34 + fork #35 merged → tagged v0.13.0 → bump PR #643 open (module half of this issue; auto-closes on merge). The lane re-run on the bumped tree confirms swap-timeout-retry / no-output-reuse = pass — the re-exposure is gone through the whole chain, not just on the branch. Remaining red on that row is service-or-refund = the #403/#258 window, owned there.
For the record: CDK hit this exact class and fixed it twice. cdk#1278 ("Fix missing try_proof_operation_or_reclaim wrapping of a swap") patched the missed call site — a swap path whose reconciliation wrapper was simply absent, our #640 verbatim. They then went structural: #1456 removed the wrapper from receive ops and #1457 slimmed the saga table, with spend operations now running as persisted sagas where the reconciliation cannot be forgotten — a spend op without its saga state doesn't compile.
Collected in #703's incident map (Class A) with the lesson spelled out: wrapper discipline rots when someone adds a call site under pressure; the durable fix shape is persisted intents + boot-time resume.
Closing the loop with a measurement: the #535 conformance lane re-run on the RC-candidate base with the gonuts v0.13.0 pin shows swap-timeout-retry ... no-output-reuse=pass (full table on #665, run of 2026-10-07T09:20Z). The proxy observed the first swap processed-and-dropped, and the retry path no longer re-exposes deterministic outputs — v0.13.0's AmbiguousOutcomeError contract ("a state-changing POST whose answer never arrived is never re-sent; reconcile first; any regeneration from a freshly incremented counter") holds under the fault injection. The class stays pinned by the lane permanently; the structural saga-shaped fix direction remains tracked in #703.
Activity
Amperstrand commentedon Oct 5, 2026
Root cause found (code-level, verified against gonuts-tollgate v0.12.1 as bumped into this module).
The re-send is the wallet client's own transport-error retry, not net/http replay and not
swapWithRetryThree candidates were checked:
Wallet.swapWithRetry(wallet/wallet.go:885): retried — it only fires on the mint'sBlindedMessageAlreadySignedErrCode(10002), re-derives fresh blinded messages, and increments the keyset counter before sending. That path is disciplined (fix: bump gonuts-tollgate v0.7.1 → v0.7.4 for swap-counter race #266's fix); it is not the re-exposure.net/httptransport-level replay (GetBody+ reused connection): ruled out empirically — a minimal repro (POST with a*bytes.Readerbody over a pooled connection that the server kills while idle, same client shape as the fork's) shows the transport opens a fresh connection and sends the request once; a non-GET/HEAD withoutIdempotency-Keyis not replayable in Go's transport.httpPostWithRetry(wallet/client/client.go:459-474): this is it. On a transport error it sleeps a backoff and re-POSTs the identical serialized body:A dropped response after the mint processed the swap (the lane's
drop_response) surfaces as exactly such a transport error: the same derivation outputs are re-sent to the mint. The observed chain, matching every lane symptom: first swap processed by the mint → response dropped → transport error → same-body retry (the re-exposure, digests sighted twice) → mint answers 10002 for the already-signed outputs →swapWithRetry's legitimate fresh re-derivation completes the swap → payment returns1022. The lab mint tolerates the sequence, which is why payments keep working while the invariant is broken; a mint with strict duplicate-output enforcement is the brick scenario the fund-safety rules describe.Why this is the wrong retry, per the repo's own rules
AGENTS.md: "Ambiguous network results must be reconciled, never blindly retried. A swap/melt timeout does not mean failure — the mint may have processed it. On timeout, query proof/quote state before regenerating or retrying." A transport error on a money-moving POST is precisely an ambiguous outcome. The same-body retry is also applied to
/v1/mint/bolt11,/v1/melt/quote/bolt11and/v1/melt/bolt11— melt retries are the same class (a melt whose response is dropped may have paid the invoice).Fix direction (fork-level —
OpenTollGate/gonuts-tollgate, then the three-go.modbump)/v1/swap,/v1/mint/bolt11,/v1/melt/quote/bolt11,/v1/melt/bolt11). For those, an ambiguous outcome returns an error that the caller reconciles — NUT-07 checkstate for the swap outputs' inputs, quote state for melts — before any regeneration (which must use a freshly incremented counter, per the existing 10002 path)./v1/keys,/v1/checkstate, keyset endpoints) on transport errors is fine and should stay.429retry-with-Retry-Afteron POSTs is a separate question: a 429 means the mint did not process the request (it refused it), so a same-body retry there is safe in principle — but only if the mint's 429 is answered before processing, which the current single-status-check shape cannot distinguish for sure. Keep it, but document the assumption.Repro:
tests/cloud-lab/conformance/run-conformance.sh, rowswap-timeout-retry / no-output-reuse=fail(evidence under.conformance/evidence/swap_timeout_retry.log+ the fault-proxy observation log).Amperstrand commentedon Oct 5, 2026
Fix state: root-caused, fixed, and lane-proven — awaiting review/merge chain.
httpPostWithRetryre-POSTed the identical body on any transport error.AmbiguousOutcomeError, reconcile-first contract), 429 keeps the same-body retry,checkstatekeeps its full retry.v0.12.2-0.20261005164740-3c054dfc880d) —swap-timeout-retry / no-output-reuseflips fail → pass; each blinded-message digest is now sighted exactly once even when the swap response is dropped after the mint processed it.Merge chain to actually close this: fork #35 → tag → the three-
go.modbump PR in this repo → re-run the lane on the bumped main. Two follow-throughs for the bump PR: teachisAmbiguousMintOutcomeErrorinsrc/merchantto matchclient.AmbiguousOutcomeError(sorecordLateReceiveOutcomelabels the no-answer case precisely), and expect theswap-timeout-retryrow to remainfailonservice-or-refundonly — that is #403/#258, deliberately not masked here.fix(client): a state-changing POST whose answer never arrived is neve…
Amperstrand commentedon Oct 5, 2026
Chain closed through the tag: fork #34 + fork #35 merged → tagged
v0.13.0→ bump PR #643 open (module half of this issue; auto-closes on merge). The lane re-run on the bumped tree confirmsswap-timeout-retry / no-output-reuse= pass — the re-exposure is gone through the whole chain, not just on the branch. Remaining red on that row isservice-or-refund= the #403/#258 window, owned there.deps(wallet): bump gonuts-tollgate v0.13.0 — NUT-20 cdk-interop signa…
fix(wallet): no same-body retry after an ambiguous mint outcome — rep…
deps(wallet): bump gonuts-tollgate v0.13.0 — NUT-20 cdk-interop signa…
deps(wallet): bump gonuts-tollgate v0.13.0 — NUT-20 cdk-interop signa…
620: merge main forward — resolve the CHANGELOG conflict (keep both s…
Amperstrand commentedon Oct 7, 2026
For the record: CDK hit this exact class and fixed it twice. cdk#1278 ("Fix missing try_proof_operation_or_reclaim wrapping of a swap") patched the missed call site — a swap path whose reconciliation wrapper was simply absent, our #640 verbatim. They then went structural: #1456 removed the wrapper from receive ops and #1457 slimmed the saga table, with spend operations now running as persisted sagas where the reconciliation cannot be forgotten — a spend op without its saga state doesn't compile.
Collected in #703's incident map (Class A) with the lesson spelled out: wrapper discipline rots when someone adds a call site under pressure; the durable fix shape is persisted intents + boot-time resume.
Amperstrand commentedon Oct 7, 2026
Closing the loop with a measurement: the #535 conformance lane re-run on the RC-candidate base with the gonuts v0.13.0 pin shows
swap-timeout-retry ... no-output-reuse=pass(full table on #665, run of 2026-10-07T09:20Z). The proxy observed the first swap processed-and-dropped, and the retry path no longer re-exposes deterministic outputs — v0.13.0'sAmbiguousOutcomeErrorcontract ("a state-changing POST whose answer never arrived is never re-sent; reconcile first; any regeneration from a freshly incremented counter") holds under the fault injection. The class stays pinned by the lane permanently; the structural saga-shaped fix direction remains tracked in #703.