You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Table background work (expiration timer, delete/cleanup callbacks) not disposed when a database's stores are closed #1811
When a database's stores are closed and later reloaded, per-table background work is not torn down. This is pre-existing (it predates the RocksDB backup/restore work) and was surfaced by a cross-model review of that change.
Details
Two code paths close and reload a loaded database in place:
resetDatabases() in resources/databases.ts — invoked on every schema change via the ITC schema handler (server/itc/serverHandlers.js).
closeDatabase() / closeLoadedDatabases() in resources/databases.ts — the restore_backup flow and job-worker exit.
Neither disposes table-level background work before/after closing the stores. In particular:
The record-expiration eviction timer — setInterval(...) in runRecordExpirationEviction (resources/Table.ts, ~line 5537, thread 0 only) — is not stored in a handle and never cleared.
Delete-removal / cleanup callbacks registered against the audit store (addDeleteRemoval → auditStore.addDeleteRemovalCallback, resources/Table.ts ~line 5528) are not deregistered on close.
Impact
After a close, these timers/callbacks keep referencing the now-closed stores (erroring on the next tick, or silently retaining them); a subsequent reload can register a second set. Bounded, but real — most likely to accumulate on instances with frequent schema changes and tables that use expiration/eviction.
Proposed fix
Add an idempotent table-disposal method that clears the table's timers/intervals and deregisters its delete/cleanup callbacks, and invoke it before closing the stores. Wire it into both resetDatabases() and closeDatabase() so both close paths are covered.
Confirmed still fully open on current main: Table.cleanup() (resources/Table.ts:7105) exists
and is wired into the schema-recreate and discard-unpublished-table paths
(resources/databases.ts:1451, :3730, :3751), but closeDatabase()/closeLoadedDatabases()
(resources/databases.ts:2223/:2318 — the restore and job-worker-exit paths this issue names)
still don't call it.
PR #2721 (closed, superseded by #2766 for the LMDB segfault this repo just fixed) prototyped
wiring Table.cleanup() into that close path, and hit two things worth flagging for whoever
implements this issue's proposed fix:
HNSW alias-backend strand.Table.cleanup() settles/deletes a table's native-HNSW backend
unconditionally. If it's ever called per-alias (one name of a multi-alias table closing while a
sibling alias of the same table stays loaded) rather than for the whole database at once, the
surviving alias's writes silently stop reaching the index — no error, just drift. Not reachable
today (both in-tree close paths close a whole database's tables together), but a real
constraint on the fix: don't retire a table's HNSW backend from Table.cleanup() unless every
alias of that table is going away with it.
derivedIndexRuntime.close() is fire-and-forget (resources/Table.ts:7108, .catch only) —
fine for the general case, but the restore path specifically needs the physical store close to
wait for it (a job worker's closeLoadedDatabases() can otherwise drop the process-wide RocksDB
refcount to 0 before an in-flight HNSW flush finishes). Close one database alias without tearing down the store its other aliases share #2721 added an explicit await barrier
before the physical close in the restore path only, leaving the fire-and-forget behavior
everywhere else. closeLoadedDatabases() doesn't await it either way today — same gap this
issue already names for the worker-exit path.
Reference (unmerged, won't ship): branch fix/alias-close-shared-root-store at 840d14f26, resources/databases.ts's closeDatabaseWithAliases and Table.ts's awaited-stop changes.
See also #2802 (RocksDB alias handles not released together on restore_backup — the
shared-store-identity half of this problem; different gap, same close paths).
Activity
kriszyp commentedon Sep 25, 2026
Confirmed still fully open on current
main:Table.cleanup()(resources/Table.ts:7105) existsand is wired into the schema-recreate and discard-unpublished-table paths
(
resources/databases.ts:1451,:3730,:3751), butcloseDatabase()/closeLoadedDatabases()(
resources/databases.ts:2223/:2318— the restore and job-worker-exit paths this issue names)still don't call it.
PR #2721 (closed, superseded by #2766 for the LMDB segfault this repo just fixed) prototyped
wiring
Table.cleanup()into that close path, and hit two things worth flagging for whoeverimplements this issue's proposed fix:
Table.cleanup()settles/deletes a table's native-HNSW backendunconditionally. If it's ever called per-alias (one name of a multi-alias table closing while a
sibling alias of the same table stays loaded) rather than for the whole database at once, the
surviving alias's writes silently stop reaching the index — no error, just drift. Not reachable
today (both in-tree close paths close a whole database's tables together), but a real
constraint on the fix: don't retire a table's HNSW backend from
Table.cleanup()unless everyalias of that table is going away with it.
derivedIndexRuntime.close()is fire-and-forget (resources/Table.ts:7108,.catchonly) —fine for the general case, but the restore path specifically needs the physical store close to
wait for it (a job worker's
closeLoadedDatabases()can otherwise drop the process-wide RocksDBrefcount to 0 before an in-flight HNSW flush finishes). Close one database alias without tearing down the store its other aliases share #2721 added an explicit await barrier
before the physical close in the restore path only, leaving the fire-and-forget behavior
everywhere else.
closeLoadedDatabases()doesn't await it either way today — same gap thisissue already names for the worker-exit path.
Reference (unmerged, won't ship): branch
fix/alias-close-shared-root-storeat840d14f26,resources/databases.ts'scloseDatabaseWithAliasesandTable.ts's awaited-stop changes.See also #2802 (RocksDB alias handles not released together on
restore_backup— theshared-store-identity half of this problem; different gap, same close paths).
Restore a database a component holds open at the next startup with re…
cb1kenobi commentedon Nov 14, 2026
Fixed by #3920 Publish every restore_backup through one crash-safe protocol, merged 2026-11-05 for v5.4. Closing a database disposes its tables' expiration timers and cleanup callbacks, and a request made during the close gets a retryable closing error.
🤖 Generated by Claude Opus 5.5; posted via @cb1kenobi.