Skip to content

Cleanup scans of loaded tables: none under threads: 0 until a write, and only daily on threads that load the table #2831

Open

Description

@dawsontoth

Summary

Two gaps in how resources/Table.ts arms the cleanup scan (the scheduleCleanup timer that removes rows past their metadata expiry) of a table loaded from its catalog. Reads are unaffected, since they already treat a row past its expiry as absent, but the rows stay on disk.

1. Under threads: 0, no scan until the first write after startup

bin/run.ts initialize() loads, and may declare, tables on the main thread before startHTTPThreads calls setMainIsWorker(true) (server/threads/socketRouter.ts). #2823 changed scheduleCleanup to record lastCleanupInterval only once its thread has a worker index, so the first expiring write or declaration after promotion arms the scan, as the first expiring write already did on main. Nothing re-evaluates the tables loaded before promotion, though: a single-threaded node that writes nothing to such a table after a restart never scans it.

A fix would re-evaluate every loaded table's scan once the main thread becomes worker 0, for example through a databases.ts helper that the threads: 0 path calls right after setMainIsWorker(true). It has to include databases.system explicitly: ensureDB() defines it with Object.defineProperty and no enumerable, so Object.values(databases) skips it, and system tables are the ones that rely on a load-armed scan (system.hdb_oidc_token_use since #2823, and the certificate-verification tables moving to a table-level expiration).

2. A thread that only loads a table scans daily, whatever its expiration

A table loaded from its catalog calls setTTLExpiration(expirationMs / 1000) (in makeTable, around ttlFromLoad = true). The number form keeps the initial cleanupInterval = 86400000, so a thread that loads rather than declares the table scans once a day regardless of expiration; only a declaring thread gets (expiration + eviction) / 4. The catalog does not store scanInterval. Decide whether this is intentional before changing it.

Related: #1339 (per-record runtime expiresAt is not reliably evicted without a table-level expiration). dataLayer/DESIGN.md "System table bootstrap" documents both limits.

Review-Deferral: c0867d378bb34ce5 severity=minor category=correctness

Activity

  1. added theissue type on Sep 25, 2026
  2. kriszyp commented on Nov 4, 2026

    @kriszyp
    Member

    After #3938, leg 2 is fixed: a catalog-loaded table now scans every scanInterval, else (expiration + eviction) / 4.

    Leg 1 now shows in query results. Under threads: 0, after a restart with no write, search, REST list queries and SQL return a runtime TTL table's expired rows indefinitely (QA: all 200 rows still returned 41 s after expiry, on RocksDB and LMDB; on the base commit they were hidden). get and search_by_id still hide them, so "Reads are unaffected" no longer holds for queries. Worth re-checking the P3 if TTL system tables are read through query paths.

    The catalog still doesn't store scanInterval. That's accepted: only a declaration can set it, and the declaration runs again at startup.

    🤖 Generated by Claude Opus 5.5 (Claude Code); posted via @kriszyp.

  3. self-assigned this
    on Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Fields

Priority

P3

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions