Repository navigation
Description
restore_backup's target_database admission treats the source repository's pin as proof that a restoring marker on the target is its own debris. A pin can outlive its marker, so a stale pin can vouch for a marker a different repository's restore wrote — and the restore then purges a target it never claimed, bypassing the documented non-destructive guarantee.
Introduced by #2649, which added the exemption so an interrupted restore could be rerun instead of wedging. The exemption itself is right; the ownership proof is too weak.
Why the pin is not proof
The marker body is the target directory name plus a timestamp (publishRestoringMarker, dataLayer/restoreMarker.ts:274-280). It records nothing about which repository the restore came from, so admission infers ownership from the presence of this source's pin:
const ourInterruptedRestore = preexisting && readBackupPins(backupDir).some((pin) => pin.pin_id === pinId);dataLayer/rocksdbBackup.ts:1285. The pin id is derived from the target directory, so every source's pin for a given target has the same id — only which repository holds it differs.
That inference holds only while a pin never outlives its marker. It can:
releaseRestoreClaim(dataLayer/rocksdbBackup.ts:744-758) clears the marker first, then unpins. The unpin is wrapped in its owntry/catchthat logs and continues, so a failing unpin leaves the pin behind by design.- A crash between the two does the same.
Both are deliberate — the ordering exists so a crash never leaves an unloadable database whose source a purge may delete — and the stranded pin is documented as harmlessly lapsing, because pinIsLive ignores a pin whose marker is gone. That reasoning is sound for purge protection. It does not hold for the new ownership check, which asks a different question of the same artifact.
Sequence
- A restore from repository A into target T completes. It clears T's marker, then fails or crashes before unpinning. A's repository keeps a pin for T.
- A restore from repository B into T starts and is interrupted. T is left populated and marked.
- A restore from A into T is admitted:
preexistingis true (B's marker), and A's stale pin matches. T is purged and replaced with A's data.
Step 3 is exactly the case target_database promises to refuse.
Impact
Bounded, and silent. T is already unloadable and awaiting a rerun that would purge it anyway, so no healthy database is destroyed — but B's operator loses the directory their rerun was going to recover, and nothing reports it. The restore succeeds normally.
The trigger needs all three steps, so this is unlikely to be hit in the field; it is filed because the invariant is wrong rather than because it is expected.
Fix
Record the source repository in the marker body and match on that instead of on the pin. markerIsIntact already treats line 1 as the contract (dataLayer/restoreMarker.ts), so a third line is additive.
A marker written by a version that does not record the source has no line to match. The upgrade path needs a decision: fall back to today's pin check for those (preserves the rerun fix, keeps the hole for pre-upgrade markers), or refuse them (closes it, at the cost of wedging a restore interrupted across the upgrade). Settling that is most of the work here.
Both the code comment at dataLayer/rocksdbBackup.ts:1283-1284 and the pin-lifetime note in dataLayer/DESIGN.md point at the epic for this gap; they should point here once this exists.
Not covered by tests
#2649 has no regression guard for this sequence — it would need a pin deliberately stranded past its marker, then a second source's marker on the same target.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity
Restore a database a component holds open at the next startup with re…
cb1kenobi commentedon Nov 14, 2026
Fixed by #3920 Publish every restore_backup through one crash-safe protocol, merged 2026-11-05 for v5.4. A pin only vouches for the restore that owns it, so a stale pin from another restore can no longer vouch for a target marker it never owned.
🤖 Generated by Claude Opus 5.5; posted via @cb1kenobi.