Priority: P0 — research-first: the swap crash window can silently destroy received value; the correct fix shape (pre-persisted swap intents) must be designed, not improvised.

Priority: P0 — research-first: the swap crash window can silently destroy received value; the correct fix shape (pre-persisted swap intents) must be designed, not improvised.

Problem

In gonuts Receive, the swap secrets (secrets, rs) exist only in memory (swapRequestPayload). The mint consumes the input proofs the moment the swap request lands; if the process dies (or panics, or the disk fills) after PostSwap succeeds but before SaveProofs, the newly issued outputs are unrecoverable: the wallet cannot unblind signatures without rs, and the mint has burned the inputs. The customer's token is gone with no local trace of the outputs.

The same window exists in MintTokens (mint signatures → SaveProofs) and in Melt (change proofs → save).

Why it matters

This is the fund-safety gap that CDK's wallet saga exists to close (saga record with counter range + blinded messages persisted before the network call; recovery replays or restores). gonuts has no equivalent. Probability is low per-request but non-zero over fleet lifetime (OOM, power loss, flash-full — flash-full is realistic on 8 MB routers), and the blast radius is total for the affected swap. We must either close it or consciously accept and bound it — but not leave it undocumented.

Current behavior (source refs, gonuts v0.11.2)

wallet/wallet.go:822-840 — createSwapRequest builds outputs, secrets, rs in memory.
wallet/wallet.go:723-732 — swap POST then SaveProofs(newProofs); nothing durable before the POST except the counter increment (which only prevents reuse, it does not enable recovery).
CDK reference model: crates/cdk/src/wallet/swap/saga/{mod,resume}.rs — add_saga (with counter_start/end, blinded messages) before post_swap; resume_swap_saga replays via NUT-19-cached responses or /restore-style checks.

Desired invariant

At any crash point, either (a) the operation's inputs were not consumed, or (b) enough durable state exists to reconstruct the outputs (secrets + rs + expected signatures) and complete the operation at next boot. No crash window may destroy issued-but-unstored value.

Proposed scope (research deliverable first, then implementation PRs)

Research doc (docs/ in gonuts or TollGate): enumerate every in-flight monetary operation in gonuts and its crash windows (swap/mint/melt × before-POST/after-POST-before-save); for each, what durable pre-state would enable recovery; evaluate two designs:
a. Pre-persist swap intents: bbolt bucket pending_ops storing {opId, mint, keysetId, counter range, outputs, secrets, rs, inputs Ys} written in the same tx as the counter increment; removed after SaveProofs; boot-time resume: re-POST (NUT-19 replay-safe? verify mints' behavior on identical swap re-POST — cdk-mintd caches; Nutshell?) or verify inputs' spend state + mint restore API availability.
b. Accept-and-bound: document the window, add wallet balance-vs-mint reconciliation tooling to detect the loss after the fact (weaker; only acceptable if (a) proves infeasible).
Decide per-operation; implement (a) where feasible (swap first — it is the customer-facing path).
TollGate side: boot triggers resume before serving payments.

Areas / files

gonuts wallet/wallet.go, wallet/storage/bolt.go; TollGate startup.

Acceptance criteria (for the implementation phase)

Test kills the process (SIGKILL) between POST-swap and SaveProofs with a mint that recorded the swap; on restart the wallet completes the swap and the proofs appear (balance preserved).
Research doc merged with the decision and evidence.

Required tests

Crash-injection harness at the exact boundary (proxy that forwards to mint, then kills the process before response handling).

Failure-injection tests

Kill pre-POST (after intent write) → resume completes or compensates (inputs unspent → re-derive fresh range).
Kill post-POST → resume recovers outputs.
Mint lost the swap (never received) → resume re-POSTs identical request (assert mint idempotency behavior; document per-mint divergence).

Compatibility

Storage addition only; old DBs unaffected.

Dependencies

After G03's storage canonicalization lands (same file, avoid conflicts).

Out of scope

Full saga machinery beyond recovery of in-flight ops (business-level saga is TollGate-side, separate issue).

Research result: the linchpin question is answered empirically, and it lands in favor of design (a).

The experiment

Harness: two host-side gonuts wallets, a capture proxy in front of the cloud-lab's cdk-mintd 0.17.6; a token minted via cdk-cli (the standard lab path); a fresh wallet Receives it; the exact POST /v1/swap request bytes are captured, then re-POSTed verbatim twice after the original fully succeeded (inputs already consumed, outputs already signed — i.e. precisely the post-crash state a recovery path faces).

Result

request	status	signatures
original swap	200	hash `b52228d8…`
replay #1 (identical bytes)	200	hash `b52228d8…`
replay #2 (identical bytes)	200	hash `b52228d8…`

cdk-mintd 0.17.6 re-signs an identical swap deterministically (DhKE signing is deterministic for fixed Y and key): no inputs have already been spent, no 10002, byte-identical signature sets across all three responses. Recovery-by-replay is viable: a wallet that persisted the swap request (outputs, secrets, rs) before the POST can, after any crash, re-POST the identical request and unblind the same signatures.

Crash-window enumeration (gonuts v0.12.1)

op	durable today, pre-POST	crash after POST, before save	recoverable with intent record
`Receive` swap	counter range (reserved)	value destroyed — signatures only in memory	✅ re-POST identical request
`MintTokens` (line ~1129 increment → mint → save)	counter range	minted outputs lost	✅ same (re-POST /v1/mint with same blinded messages — needs the same verification as swap; deterministic signing suggests yes, verify)
`Melt` change (post-fd19764: range reserved pre-POST)	blank-output range reserved	change signatures lost	✅ re-POST /v1/melt/bolt11 identical — melt side effects (invoice paid) are external; checkstate (`CheckMeltQuoteState`) already exists as the reconcile primitive

Design (a), concretely

Intent record — bbolt bucket pending_swaps, key opId (uuid), value \{mint, keysetId, counterStart, counterEnd, outputs (B_), secrets, rs, opType, requestBytes\}, written in the same bolt transaction as the counter increment (bolt Update is already the wrapper — one tx = atomic intent+reservation; this is the property that makes the invariant hold). Deleted only after SaveProofs commits.

Resume at boot (TollGate startup, before serving payments — matches this issue's TollGate-side scope; also naturally hookable next to #507's reclaimer loop for periodic retries):

Load each intent.
Re-POST requestBytes verbatim.
200 + signatures → unblind with persisted rs → SaveProofs → delete intent.
Non-200 / connection error → checkstate the input Ys: inputs UNSPENT → swap never landed → safe to abandon (delete intent; the customer's token was never consumed — they retry) or re-execute from the reserved range; inputs SPENT but re-POST refused → mint divergence: log + surface as destroyed-value accounting (the bounded-accept fallback), never retry blind.
Rate-limit replays (exponential backoff) so a down mint isn't hammered.

Residual, irreducible window: essentially none for the swap itself (any crash point maps to step 2 or 4); what remains is mint-side keyset retirement between crash and resume (same exposure as any mint loss) and mint-implementation divergence (handled by 4).

Test plan → acceptance criteria mapping

Kill pre-POST, post-intent (issue's case): inputs unspent → abandon-or-reexecute. ✔ step 4.
Kill post-POST pre-save: now verified recoverable by replay (the experiment above is this case, manually). Needs the automated SIGKILL-at-boundary harness the issue asks for: the capture-proxy harness built for this experiment generalizes — a proxy that forwards then kills the client process before response handling.
Mint never received the request → step 4 unspent branch.
Per-mint divergence → step 4 spent-but-refused branch, exercised with a stub mint that rejects replays.

Open questions (blocking nothing, worth one pass each)

Nutshell replay behavior — unverified (testnut is https; no MITM in this harness). The signet-lane work tracks an http nutshell sidecar for the lab; run the same capture experiment there before trusting replay on nutshell mints.
MintTokens/melt replay — deterministic-signing argument says yes; verify with the same method.
cdk-mintd version sensitivity — 0.17.6 verified; note as a dependency assumption.

Footnote

During harness construction, gonuts Send against cdk-mintd 0.17.6 errored Signature missing or invalid (mint-side) in our setup — not chased (out of scope here; the lab's payment paths don't use it). Worth its own issue if it reproduces cleanly.

Research evidence for the pre-persisted-intent design (unattended run; gonuts v0.12.1 local source + cashubtc/nuts@`8f244be`, cdk@b7cb458, nutshell@e9dd82a)

1. The mint-side truth: identical swap re-POST is NOT a recovery mechanism

NUT-03 specifies no replay safety — inputs are verified and invalidated, full stop. Replay safety exists only via NUT-19 cached responses, which is optional: key = method+path+payload, only 200s cached, TTL mint-chosen, announced via NUT-06 nuts["19"] (a wallet must read mint info to know replay is even possible).
cdk-mintd: /swap is always NUT-19-announced, but the default backend is in-memory with a 60 s TTL (cdk-axum/src/cache/mod.rs:74,101) — a narrow window. On cache miss/expiry the mint returns 11001 "Proofs already spent" (TokenAlreadySpent, legacy code 10002 — gonuts' matcher should accept both).
Nutshell: the @redis.cache() decorator on /v1/swap is a passthrough unless mint_redis_cache_enabled (default False, Redis required); uncached re-POST → ProofsAlreadySpentError 11001.
Verdict: recovery must not re-POST as its primary path. Re-POST is a best-effort fast path, valid only when info announces NUT-19 for /v1/swap AND within TTL.

2. The spec-endorsed recovery pair: NUT-07 + NUT-09 (both mints implement)

/v1/checkstate (NUT-07): answers "were my input proofs consumed?" (UNSPENT/PENDING/SPENT by Y) — the did-it-execute signal for resume logic. Note: it covers input proofs only; output blind-message state is not exposed here.
/v1/restore (NUT-09): the mint stores BlindedMessage+BlindSignature for every signature it ever issued and re-returns them when the wallet re-submits the same BlindedMessages. The spec explicitly endorses this "for recovering the response of an interrupted swap request (see NUT-03)". This is the mechanism that recovers outputs the wallet can still name.
The gonuts-critical nuance: restore returns the signatures; unblinding still requires the rs. CDK gets re-derivation for free (NUT-13 deterministic secrets, PreMintSecrets::restore_batch(seed, counter_start..end)). gonuts uses random in-memory rs — so the intent record MUST persist secrets + rs (blinded messages alone are insufficient). This confirms the issue's proposed field list {opId, mint, keysetId, counter range, outputs, secrets, rs, inputs Ys} is exactly right, and minimum.

3. gonuts v0.12.1 source map (every window + the integration points)

The windows (POST-succeeds → SaveProofs): swap at wallet.go:721-732 (Receive) and ~794 (ReceiveHTLC), swapToTrusted ~935; MintTokens 348→415; melt change 1118→1201. Counter increments (durable pre-POST) at six sites: 420, 721, 794, 906 (the tollwallet: service restart permanently breaks swaps — 'Duplicate outputs' against any mint with prior swap history #480 retry regeneration), 1023, 1129 — the intent write belongs in the same bbolt tx as each, and the 906 retry must REPLACE the intent row (first attempt's outputs were rejected; only the retry's secrets matter).
Melt already has half a saga: inputs move to pending (durable) before the POST and the persisted quote reconciles (Unpaid→restore proofs, Paid→delete). The gap is the change outputs' secrets/rs (in memory at 1118) — same intent-record fix, different bucket flow.
Same-tx feasibility: IncrementKeysetCounter (storage/bolt.go:623) is a single bolt.Update closure — adding a pending_ops bucket Put inside it (or a sibling call composed in one tx) is mechanical.
CDK reference shape (confirming the design): saga persisted with state+counter range+blinded messages before post_swap (optimistic-locked update at SwapRequested); resume order = DB fast-path check → identical re-POST (NUT-19 best-effort) → checkstate inputs → SPENT: /restore with the persisted blinded messages → UNSPENT: compensate (release inputs) — mint unreachable: leave for retry.

4. Verdict

Design (a) — pre-persisted swap intents — is feasible and correctly shaped: same-tx intent+counter write, fields as the issue lists (secrets+rs mandatory), resume in the CDK order (re-POST demoted to best-effort), extend melt's existing pending-proof mechanism for the change-output window. Design (b) accept-and-bound remains the fallback only if the storage tx proves conflict-prone with G03's canonicalization. The crash-injection harness the issue specifies (kill between POST and SaveProofs against a proxy mint) can assert: resume-with-persisted-rs completes the swap (balance preserved); resume-without-persisted-rs documents the accepted loss.

fix(cloud-lab): lab.sh taps need fork — one connection was all they s…

Melt-path empirics complete — the #497 enumeration is now fully answered empirically.

Method: same capture-proxy harness (in-process, in front of the lab's cdk-mintd 0.17.6); a wallet mints 128 sats (via the NUT-20 legacy path), melts 21 against a fresh FakeWallet invoice (bech32-mutated so the mint has never settled it), deriving NUT-08 change outputs; the exact /v1/melt/bolt11 bytes are captured, then re-POSTed verbatim twice after the melt fully succeeded (invoice paid, quote PAID, change issued).

request	HTTP	state	change signatures
original melt	200	PAID	present (1 change output)
replay #1 (identical bytes)	200	PAID	identical
replay #2 (identical bytes)	200	PAID	identical

Response sha256 byte-identical across all three (dfa52b9a…). The critical behavior: cdk-mintd replays the melt response idempotently — including the change signatures — rather than refusing an already-paid quote. The external side effect (invoice payment) is not repeated.

Implication for the design

Recovery-by-replay is sound for the melt change too: persist {melt request bytes, change outputs, secrets, rs} atomically with the blank-output counter reservation (which fd19764 already writes pre-POST); on crash-after-paid, re-POST returns the same change signatures, which the persisted factors unblind. The inputs' pending marking (already pre-POST) plus the replay together close the melt window.

Full enumeration status: swap ✅ (#31 evidence, #33 machinery), mint ✅ (#33, byte-identical ×3), melt ✅ (this comment — implementation follows the #33 pattern). Every in-flight monetary operation in gonuts now has verified deterministic-replay semantics on the deployed mint implementation.

The final #497 deliverable is done: the SIGKILL-at-boundary harness, verified live at the TollGate layer — this issue's acceptance criterion, verbatim: *"Test kills the process (SIGKILL) between POST-swap and SaveProofs with a mint that recorded the swap; on restart the wallet completes the swap and the proofs appear (balance preserved).""

Live run against the intent train (gonuts #31/#33 + the boot-resume wiring, stacked via replace):

step	evidence
mint signed the swap	killer-proxy marker `/v1/swap` (every retry response swallowed — the wallet library's 5-attempt connection ladder held open)
SIGKILL	`docker kill` at the marker, tollgate mid-wait
restart → boot resume	`wallet: pending op 87eb08e9… recovered 50 sats from http://killer:8085`
wallet healthy	post-recovery fresh payment → 200

The harness itself is TMBG PR #569 (run-crash-injection.sh + killerproxy.py): a release-gated killer service in the compose network, self-isolating run (own project/subnet/names), driver script with the two assertions. Against pre-intent binaries the recovery assertion fails by design — that is the demonstration that the window costs value without the machinery.

#497 is now complete end-to-end: empirics (swap/mint/melt all deterministic-replay on cdk-mintd 0.17.6) · fork implementation (#31 swap intents, #33 mint+melt intents, stacked on #32) · TollGate boot wiring (held branch, unblocks on tag) · crash-injection harness (#569). The issue can close when the train merges and the harness passes on main.

test(cloud-lab): #497 crash-window harness — SIGKILL between mint-sig…

fix(cloud-lab): lab.sh taps need fork — one connection was all they s…

docs(release): the measured conformance state and the min_steps spec …

release: v0.6.0-rc1 — feature freeze (merge last) (#659)

620: merge main forward — resolve the CHANGELOG conflict (keep both s…

test(cloud-lab): run-scoped lab runner + shared-host runbook (#557)

test(cloud-lab): #497 crash-window harness — SIGKILL between mint-sig…

Cross-linking the fund-safety program issue #703 (incident map + pre/post-release plan), plus two research finds that sharpen this issue's fix shape:

The saga layout this issue cites is current on cdk main (verified 2026-10-07): crates/cdk/src/wallet/{swap,send,melt,issue,receive}/saga/ + wallet/saga/mod.rs, mirrored on the mint side (mint/{swap/swap_saga,melt/melt_saga}/). Mechanics from source: prepare() reserves the input proofs, reserves the derivation-counter range in one atomic increment_keyset_counter call, and persists the saga record (add_saga) before post_swap; receive/saga/resume.rs resumes replay-first — re-POST the original request (a NUT-19-cached mint response returns the signatures immediately), fall back to NUT-09 /restore only if replay fails, else compensate. So design (a) in the research doc has a living reference implementation, not just a description.
CDK hit this same class and their fix history is instructive: cdk#1278 fixed a swap path missing its try_proof_operation_or_reclaim wrapper (our wallet: swap-timeout retry re-exposes derivation outputs (measured by the #535 conformance lane on current main) — the #257/#266/#480 brick class #640 verbatim); #1456/#1457 then replaced wrapper idioms with the persisted saga, so a forgotten reconciliation call site no longer compiles away the safety.

Also relevant: cashu-audit's issues/interrupted-operation-recovery.md (Amperstrand/cashu-audit, GH-tracked upstream draft) proposes a NUT for operation-recovery semantics — state machines across crash points, idempotency keys, restore obligations. Whatever design doc comes out of this issue should feed that draft; TollGate is the interoperable-payment-router case study the spec gap hurts most.