Repository navigation
Degraded-mode wallet factory bypasses wallet.backend (gonuts wallet constructed under backend=cdk) #1157
Description
State-machine audit finding (layer L4/L1, split-brain-wallets class), context measured 2026-11-01 during the PR #1155 e2e work:
The path. When the running merchant observes all mints unreachable, the runtime downgrade fires (main.go registerReachableSetChangedCallback → 'All mints unreachable — downgrading to degraded mode' → merchant.NewMerchantDegradedFromFull → NewMerchantDegradedWithWallet(..., DefaultWalletFactory, ...)), and DefaultWalletFactory (merchant_degraded.go) hardcodes tollwallet.New — the legacy in-process gonuts constructor — regardless of the wallet.backend selection. The same bypass exists on the boot-time degrade paths (merchant.go, both the no-reachable-mints and the wallet-init-failed branches).
Consequence under wallet.backend=cdk. While degraded, the service constructs a gonuts wallet at walletDirFor(configManager) and serves payments from it once mints return (WireRecoveryTrigger upgrades back to a full merchant only after the aggressive probe fires). Any payment landing in the mint-returned-but-still-degraded window is received into the gonuts wallet: value splits silently away from the operator's cdk-walletd balances — the split-brain class (two wallet instances, one mint).
Why it is narrow today (measured, not assumed). A sidecar (cdk-walletd) outage does NOT reach this path anymore: it dies at the pre-flight decode RPC and returns payment-processing-unavailable before Receive's mint-health classification (fixed in d4ff8e6). The degrade requires the mint itself to be unreachable (Receive error → MarkUnreachable → set empties), and while the mint is down the gonuts wallet cannot receive either. The window is mint-returns-while-still-degraded, seconds wide under the aggressive retry loop — narrow, but real, and it silently violates the operator's backend choice.
Fix shape. The degraded merchant's wallet factory must route through the same selection seam as the full merchant (openServiceWallet): for backend=cdk, degraded mode holds a SidecarWallet client (cheap to construct, reconnects per call) or runs wallet-less (WalletLoaded=false) — either is defensible; constructing a second wallet IMPLEMENTATION is not. The boot-time branches and the runtime downgrade both need it.
Evidence. Full trace of the chain in the 2026-11-01 handoff section (/root/handoffs/cdk-wallet-migration-phase0.md); the clean-repro attempts and why they could not trigger it (decode-first + mint probe keeps the reachable set nonempty) are documented there too. Related lineage: #1139 (the seam's first bypass, fixed by #1155's openServiceWallet).
Activity
fix(merchant): degraded-mode wallet factory respects wallet.backend (#…
Amperstrand commentedon Oct 31, 2026
Fixed on PR #1155 (dd2ddd7): walletFactoryForBackend routes all three degrade paths through the selection seam; cdk + dead socket = wallet-less degraded (fail closed), default = legacy factory byte-identical. RED-first test TestDegradedWalletFactoryRespectsBackendSelection. Will close when #1155 merges.
Amperstrand commentedon Nov 5, 2026
Fix re-landed on current main as PR #1200 (the #1155 branch that carried dd2ddd7 is 20 commits into campaign territory and CONFLICTING with main, so the slice was extracted and adapted to main's mintUnits plumbing).
What #1200 ships for this issue:
walletFactoryForBackendroutes all three degrade paths (both boot branches + the runtime downgrade viaNewMerchantDegradedFromFull) through thetollwallet.OpenWalletseam —backend=cdk+ dead socket constructs no wallet (fail closed, wallet-less degraded, honestErrSidecarNotConnectedlog naming the class), the default spellings keepDefaultWalletFactorybyte-identical, and unknown backend spellings are refused by the seam rather than silently treated as the default.Mutation-checked against pristine
origin/main: the runtime-downgrade pin fails there with the exact finding (log showsDegraded mode: offline wallet loaded successfully— a gonuts wallet constructed underbackend=cdk); green on the PR.make go-battery19 modules green.fix(merchant): DefaultWalletFactory honors wallet.backend selection (#…
Amperstrand commentedon Nov 5, 2026
Fix on PR #1203 (
df3f7f38): DefaultWalletFactory routed through tollwallet.OpenWallet (the selection seam). Falls back to in-process when the sidecar is unreachable or the backend is unknown — a wallet-selection issue must never hard-fail the degraded path. Degraded suites green under -race.Amperstrand commentedon Nov 5, 2026
Review finding: the env var seam is inert — TOLLGATE_WALLET_BACKEND is not set by anything. The fix on #1203 routes through OpenWallet (correct structure), but
os.Getenv("TOLLGATE_WALLET_BACKEND")always returns "" because no setup script or config system exports it. On a real router with wallet.backend=cdk, the degraded path still uses gonuts.What's needed to make it real: the init script (or the config manager) must export TOLLGATE_WALLET_BACKEND from the uci config at service start. This is a small packaging/config change, not a merchant change — the routing code is correct. Filing this so it isn't forgotten when the migration's config seam lands.
review: decision record corrections + safe type assertion
revert: remove the #1157 fix — deferred to #1200 which does it properly
Amperstrand commentedon Nov 5, 2026
Definitive fix: #1200 — wires wallet.backend through the config system (schema + fail-closed on dead sidecar + typo refusal + tests). Our earlier env-var attempt on #1203 has been removed as inferior. Closing — #1200 is the real fix and is ready for review.