[kriszyp]

⊙ Problem

Since #4211 moved rocksdb-js to 2.14.0, Unit Test has failed on every main commit. rocksdb-js 2.14 refuses to open a column family with the verification table (the freshness check behind the record cache) while another handle in the process holds it without one. A test left one open, breaking later files. Changing a plain index to HNSW on a live table hit it too: table() threw, even after a restart.

💡 Solution

• On that refusal, openIndex reopens an HNSW index without its record cache and warns. Reads stay uncached until restart. Primary stores still throw.
• A thread that only reloads the catalog now switches index kind; before, its writes skipped the HNSW graph.
• The test's opted-out table gets its own database, dropped after the file.

⚖️ Alternatives

• A separate column family for the HNSW index — changes on-disk naming; an older reader would open the plain index as the graph.
• Refuse the change until restart — turns a working 2.13 schema change into drop, restart, re-add.
• Close the old plain handle first — other workers hold their own, and rocksdb-js also refuses while older transactions run.

🤔 Should rocksdb-js accept a late opt-in instead?

If rocksdb-js converted open handles on an opt-in, the uncached fallback could go. That is a native write-path change; main needed to be green first.

🤔 Uncached HNSW reads until a restart

Only an index whose type changed while Harper ran is affected (#4257). The cost is not measured.

✅ Verification

• The three resources files that cascade on CI, in one process: 3 failing on main, 29 passing here.
• verificationTableParticipation.test.js: the type change fails on main; the reload case fails without the initStores change; a fresh HNSW index keeps its cache.
• CI, on this branch merged with main at 3b9072d: resources-core-o-z passes in the native and build legs; main's same job fails 59, 58 of them on ConflictRetryTable/'s refusal. The remaining reds fail the same tests on main: components and resources-core-a-n (Two branch unit tests fail on rocksdb-js 2.14: a close now reports RocksDB's Close() status, and a live branch directory is copied while it compacts #4252), and Integration 1/6 (Bun), 6/6 (Bun) and 6/6 (uWS HTTP). 2/6 (Windows) fails native-plane-coverage (HNSW write lag), as on four other branches' runs in the same hour.
• PR QA: queued qa-pr-harper-4260-3d645bb @ 3d645bb

Gates and the 2.13 A/B

• Each cause was checked against 2.13.0 swapped into node_modules, the tree otherwise unchanged: the cascade and the type change pass there.
• test:unit:resources at this head: 7626 passing, 3 failing on a loaded box (originClosedFloor, recordLockCoordinator, uniqueLocalLogKeys, all timing); their 298 tests pass when the files run alone, and on main's CI.
• test:unit:main:components: fails only Two branch unit tests fail on rocksdb-js 2.14: a close now reports RocksDB's Close() status, and a live branch directory is copied while it compacts #4252's test.
• The LMDB resources run hit the known intermittent segfault; the file it crashed in passes alone.

Docs: release note; no reference page describes the record cache.

Closes #4222

_{🤖 Generated by Claude Opus 5.5 (Anthropic, Claude Code dev agent); posted via @kriszyp.}

Release target: v5.4 — rocksdb-js 2.14 is only on main

Related PRs: #4250 overlaps (both edit initStores' catalog loop in resources/databases.ts), 4 others independent
Complexity: complicated

Deferred findings

• 105c576cb7142ca9 major data-integrity — resources/databases.ts — Catalog-only threads write plain entries into the HNSW column family until they reload the catalog — filed A thread that has not reloaded the catalog writes to an index through its old store while another thread rebuilds it #4256
• aad7c8adb2d04da1 minor performance — resources/databases.ts — indexFor never closes the superseded plain wrapper — filed An index changed from plain to HNSW while Harper runs reads without its record cache until restart #4257
• 9972708621c19c54 nit reliability — resources/databases.ts — Fallback treats rocksdb-js's temporary pending-transaction conflict as permanent and logs a wrong cause — filed An index changed from plain to HNSW while Harper runs reads without its record cache until restart #4257
• 13a1cc5543093669 nit test-coverage — unitTests/resources/verificationTableParticipation.test.js — The fallback's warning is promised by the release note but not asserted — filed An index changed from plain to HNSW while Harper runs reads without its record cache until restart #4257
• 91fa3e288683192b nit correctness — resources/databases.ts — LMDB catalog reload keeps a stale wrapper after a plain/HNSW type change — filed A thread that has not reloaded the catalog writes to an index through its old store while another thread rebuilds it #4256

Origin — the dispatch brief this PR was written from

Turn harper main green on rocksdb-js 2.14: open column families consistently with verificationTable

_{Dispatch: task proposal-kriszyp-harper-4222-rocksdb-2-14-main-red-dd2eff · queued by unknown · ran by claude/opus/xhigh · worker kzyp-xps-1}

_{Review-Coverage: authored=unknown; ran=none; rounds=1 @ 1f1ceaa}

_{Review-Attention: deep ~25m (raised: unmeasured diff) @ 1f1ceaa}