Repository navigation
Description
Background
PR #2470 (fix/integration-flake-triage) spent ~45 rounds of independent review building a fail-closed cross-worker database-drop protocol (sig 2 of that investigation): a reused .restoring lifecycle marker, an ITC close_database signal, a process-wide registryStatus() closure check, and a 409 refusal when a handle survives the broadcast — closing a RocksDB LOCK-held-forever wedge (open racing a destroy) and a long tail of handle-leak defects the review rounds found along the way.
While #2470 sat in review, origin/main independently merged its own solution to the same problem in PR #2616 ("Activate native full-text indexes with shared lifecycle coordination", merged 2026-09-26): resources/databaseDropPreparation.ts's claimDatabaseDropPreparation()/DatabaseDroppingError (409), per-root drop locks, and registryStatus()-based closure checks of its own — built for a different original purpose (full-text derived-index teardown) but covering the same destructive-DDL-vs-process-global-native-handle problem.
Per-maintainer decision, #2470 is being recut: sig 1 (already fixed on main), sig 3 (test fix), and sig 4 (Windows process-tree PID recycling) landed independently as #2890. Sig 2 — the drop protocol itself — is not being ported; #2616's protocol is the one going forward. This issue is the remaining piece.
The task
Audit origin/main's dropDatabase()/resumeIncompleteDatabaseDrop()/databaseDropPreparation.ts (from #2616) against the specific defects #2470's ~45 review rounds found and fixed in its own (now-abandoned) protocol, and port forward only what's actually missing. This is an audit-then-implement task, not a revival of #2470's marker-based architecture.
Confirmed already present in #2616 (read directly against source, not assumed):
- Fail-closed 409 via
registryStatus(), checked both before and after the destructive close (resources/databases.tsdropDatabase()) - Crash recovery via
resumeIncompleteDatabaseDrop()with a drop marker (databaseDropMarkerPresent) - Per-root drop locks and blob-path resolution at drop time
Not yet verified either way against #2616 — the specific gaps to check, each with the #2470 commit/round that fixed it as a starting point:
- Index-store handle-leak-on-throw ordering: open-new-before-closing-old, and closing only after the new handle is published to the live table's index map (not just successfully opened) — on both the kind-change-reopen and first-time-open branches (rounds 10–13 of Stop a recycled Windows PID from wedging deploy_component and release dropped databases on every thread #2470)
- LMDB close-before-unlink ordering, and whether LMDB's own drop path shares any of this protocol at all in Activate native full-text indexes with shared lifecycle coordination #2616 (round 6–7 of Stop a recycled Windows PID from wedging deploy_component and release dropped databases on every thread #2470)
- Iterative/streamed directory removal for a dropped database's files vs. a single bulk
fs.rm({recursive:true})occupying a libuv threadpool slot for the whole delete on a large tree (round 13–14 of Stop a recycled Windows PID from wedging deploy_component and release dropped databases on every thread #2470) — Activate native full-text indexes with shared lifecycle coordination #2616'sdestroyIncompleteDatabaseRoot()currently uses the bulk form; unclear whether that matters given Node's native implementation differs from the pattern Stop a recycled Windows PID from wedging deploy_component and release dropped databases on every thread #2470 moved away from - The on-demand-open guard's marker-recheck race (a drop marker replaced by an incoming restore in the gap between reading marker kind and marker state) (round 10, 14 of Stop a recycled Windows PID from wedging deploy_component and release dropped databases on every thread #2470)
Table.cleanup()(timers, callbacks, reclamation handler) ordering relative tocloseDatabase/store teardown (round 6 of Stop a recycled Windows PID from wedging deploy_component and release dropped databases on every thread #2470)- The drop marker's own deletion-target manifest (recording exact blob roots at drop time rather than re-deriving from live, possibly-since-changed
storage.blobPathson crash recovery) — a data-loss scenario Stop a recycled Windows PID from wedging deploy_component and release dropped databases on every thread #2470 added a versioned manifest line for; check whether Activate native full-text indexes with shared lifecycle coordination #2616's marker format has an equivalent
Why this matters (if gaps are real)
The original driving bug: an open racing a destroy left a RocksDB LOCK held forever after a crash — every subsequent database open on that path fails, job workers die at boot. Structural trigger (any concurrent drop + rescan/open under real load), unbounded (no self-recovery without operator intervention), and whatever fraction of the handle-leak tail isn't independently closed by #2616 reopens some slice of that same failure class.
Scope note
Not a blind port. #2470's marker-based architecture is superseded — the ask is specifically: does #2616 already close each gap above, and if not, what's the smallest patch on top of #2616's existing protocol (not a parallel one) to close it. Expect this to need the planning-review gate (prepush-review.mjs --mode plan) before implementation, since it's core shared database-lifecycle code.
Activity
kriszyp commentedon Oct 23, 2026
Closing: the six-gap audit is done and Remove a dropped LMDB database's file only after its environment has closed (#3321) fixed the last open correctness gap. It merged to
mainon 2026-10-23 (v5.4).Table.cleanup()orderingGap 3, accepted:
fs.rmstarts removing every file at once, which can occupy all four libuv threadpool slots. In a measurement, a concurrentstatstalled for 404 ms while a 25.6 GB tree (400 × 64 MB files) was deleted, against 0.2 ms with streamed removal. Harper only usesfs.rmfor detached roots and crash recovery; normal drops go through nativedestroy()orunlink. The stall is bounded and recovers on its own, and the normal RocksDB drop path already blocks the owner thread for the same deletion. If it ever shows up in practice, the fix is to switch those twormcalls to a streamed loop.🤖 Posted by Claude Code (Claude Opus 5.5) on behalf of @kriszyp.