[kriszyp]

⊙ Problem

A table's saved expiration and eviction had four gaps:

• Removing one from @table, or setting 0, never cleared it. Records kept expiring, after restarts too.
• A negative value on a new table failed later, naming no table. create_table ignored eviction.
• A thread that adopted an eviction-only TTL never scanned for expired rows.
• Under threads: 0, catalog-loaded tables were not scanned until a write.

🤔 Clearing is a behavior change

An operator who relied on a removed expiration staying in effect must declare it again. Records already written keep the expiry they were given.

💡 Solution

• A bad TTL value fails with a 400 naming the table and field, before anything changes.
• @table and defineTable clear what they omit. table() callers such as add_attribute keep it.
• A TTL from sourcedFrom options survives a schema without one, in isolated applications too.
• One rule sets the scan interval however a TTL arrives.
• Under threads: 0, scans start once components load.

⚖️ Alternatives

🤔 A thread keeps the TTL its own declaration set

A thread whose schema declared a TTL ignores a later value another thread saves, until it declares again. Every worker running the same schema re-declares on restart, so this shows only when threads declare differently (#4240).

🤔 A schema 0 counts as declaring no TTL

expiration: 0 leaves an application-set TTL in place, as omitting it does; declareTable cannot tell them apart (#4253).

🤔 A peer's invalid saved TTL fails its create

An older peer's negative or non-numeric TTL fails the create, naming the field; a local load drops it with a warning (#4254).

🤔 A cleared table keeps a daily scan

Rows written before a clear still carry their expiry, so the table keeps a daily cleanup scan, marked expiryScan on its catalog row. The mark is never removed; the alternative was leaving those rows readable and stored forever after a restart.

Other options weighed

Framing-Verdict: better-alternative-exists — resolved by adopting the reviewers' alternatives (strict input types, keeping a saved value the declaration omits, no new cadence helper, arming after components load) and overruling two:

• Making the main thread worker 0 before startup under threads: 0: startup runs upgrade directives, compactOnStart and migrateOnStart first, and scans would run during them.

• Sending a cleared TTL to cluster peers: an existing table never takes a peer's TTL, set or cleared.

• Validate only in the operations API and GraphQL: misses defineTable and application table() calls.

• Treat only an explicit 0 as clear: removing a value from a schema would still do nothing.

• Arm threads: 0 scans right at promotion: a stale saved TTL would start evicting while the schema that removes it is still loading.

🔧 Changes

Declaration	Before	After
@table / defineTable without expiration	saved value kept	cleared
expiration: 0 or null	saved value kept	cleared
table({ expiration }) without eviction (add_attribute, cluster events)	eviction kept	eviction kept
expiration: -1, 'soon', true	-1 failed naming no table; 'soon' and true were accepted	400 naming table and field, nothing changed
create_table with eviction	ignored	saved

Files

• resources/Table.ts: ttlSecondsOf validates and savedTtlSeconds reads saved values; setTTLExpiration validates every value first, leaves the part of the TTL application code set, as applicationTtl records it, and holds the one cadence rule; a clear of a scheduled TTL sets retainExpiryScan (declared from the catalog's expiryScan); _adoptCatalogExpiration adopts cleared and eviction-only rows and the reload's expiryScan; _armCleanupScan; makeTable validates any loaded TTL that is not absent or zero, so a peer's bad value fails its create; resumeMaintenance keeps the retained scan; scheduleCleanup drops its zero-interval exit, which nothing reaches now.
• resources/databases.ts: ttlChanged and loadedTtlSeconds, which initStores uses to load an invalid saved value as absent with a warning; initStores passes expiryScan to a new class and to a reloaded one; the declaring thread applies a clear; persistTableSettings marks the row expiryScan; armTableCleanupScans; TableDefinition allows null.
• server/threads/socketRouter.ts calls armTableCleanupScans after startServers() under threads: 0.
• dataLayer/harperBridge/ResourceBridge.ts: create_table forwards eviction. dataLayer/schemaDescribe.ts: describe_table shows no expiration once no TTL is in effect; an eviction-only table still shows 0s.
• Docs: schema.md, operations.md, the 5.4 release note. Design notes: resources/DESIGN.md, dataLayer/DESIGN.md (drops the two TTL limits this fixes), resources/MAP.md.

Tour: which declarations set, clear or keep a TTL, which thread holds which, and when scans arm

Product and architecture tour

What a declaration does to a saved TTL

Which declarations set, clear or keep a table's saved TTL?

declareTable checks each value with ttlSecondsOf before anything is opened or saved, so a bad value leaves no table and no catalog row behind. @table passes null for what it omits, and so does defineTable. persistTableSettings deletes a cleared field. A table() call that names only one field keeps the other as the row it writes holds it, re-read under the schema lock — look hardest here: that row is also what the declaring thread applies.

Which TTL each thread holds

Which value wins when threads disagree?

A thread that declared the table, or whose application code set its TTL, keeps that TTL until it declares again.
Every other thread, and every restart, takes the saved value.
A schema that declares no TTL does not clear one application code set, in an isolated application too; it does clear a TTL a schema set later.

When the cleanup scan arms

When does each thread start reclaiming expired rows?

From load to scan

1. Load — a saved TTL derives the interval: scanInterval, else (expiration + eviction) / 4, else daily.
2. Adopt — a reload that finds a changed row, cleared or eviction-only included, goes through the same setter.
3. Clear — stamped rows keep their expiry; the scan moves to daily and the row records expiryScan.
4. Promote — under threads: 0, after components load, armTableCleanupScans arms each loaded table that has a TTL or expiryScan.

✅ Verification

• QA-5424 (an isolated application's schema zeroing a sourcedFrom TTL): four new unit cases fail at 695e719 and pass here, on RocksDB and LMDB.
• The new integration test passed 12/12 at 695e719 and fails on main; this head's change is unit-covered, and CI re-runs it.
• 28 new unit cases green on both engines; the TTL integration set passed 20/20 at 695e719.

Test details

• An isolated application's @table without a TTL kept zeroing a TTL from sourcedFrom options (QA-5424). Four new unit cases cover it: the isolated table factory with sourcedFrom first, both owner kinds, a schema TTL that replaced an application's, and a number-form call over a schema TTL. All four fail at 695e719 and pass here, on RocksDB and LMDB.
• New table-expiration-config.test.ts (threads: 0 suite): RocksDB and LMDB, four workers and threads: 0, with restarts. 12/12 passed at 695e719; it was not re-run locally on this head (no loopback aliases on the worker), so CI runs it. On an origin/main build every check fails except the four-worker runtime-table reclaim, which Serve id and id-plus-indexed-attribute selects from the index entry instead of reading every record #3938 already fixed.
• 24 new unit cases in ttlCleanupInterval.test.js and tableExpirationPersistence.test.js, green on RocksDB and LMDB; the 13 written first all fail on main. createTableCatalogOrder.test.js now fails inside makeTable through a peer definition, since a local one is rejected earlier.
• TTL integration set (runtime-table-ttl-eviction, record-caching-ttl-eviction, apiTests/ttl, the new test): 20 pass, 0 fail.

Test fixtures

Its fixture's TtlControl pins declarations to one worker so the scan owner learns TTLs only from the catalog; schema.graphql and config.yaml declare the TTL the test removes; NegativeTtl (config) checks the failure names table and field.

The three four-worker cases that must reach a chosen worker skip Windows and Bun, which do not spread connections across workers.

Full test:unit:resources run

6669 passing. The same 101-test ConflictRetryTable/ cascade and originClosedFloor clock failure occur on a main build. Three replica-pressure timing assertions failed in the full run; interleaved, replicaStorageReclamation.test.js passed 5/5 on both branch and main.

Performance: no record read or write path changes. Validation runs per declaration; socketRouter.ts adds one pass over loaded tables at threads: 0 startup; a cleared table keeps one daily metadata scan.

PR QA: Does an isolated application's table keep the TTL its sourcedFrom options set, whichever plugin loads first, across restarts? — queued @ 51a5a77, qa-pr-harper-4250-51a5a77

Docs: in this PR.

_{🤖 Generated by Claude Opus 5.5 (Claude Code); posted via @kriszyp.}

🤖 Generated with Claude Code

Release target: v5.4 — a behavior change documented in the 5.4 release notes; the gaps are also on v5.3, where a backport is a customer-need call

Related PRs: #2155 overlaps (both change scheduleCleanup; this removes its zero-interval exit, which that PR rewrites), 6 others independent
Complexity: complicated

Deferred findings

• b8406dd465c94218 minor correctness — resources/Table.ts — explicit schema zero preserves application-set TTL — filed A schema TTL of 0 leaves an application-set TTL in place, like an omitted one #4253
• 61d6fd7c994922b5 minor reliability — resources/Table.ts — peer definition with legacy invalid TTL now fails table create; catalog load tolerates same value — NOT FILED
• ac1c5dc3bd820ff9 nit correctness — dataLayer/schemaDescribe.ts — eviction-only table described as expiration 0s, eviction unreported — filed describe_table reports expiration 0s for an eviction-only table and never reports eviction #4242
• 9ee5a87fa17da5dd nit maintainability — server/threads/socketRouter.ts — suggest comment on why multi-worker path needs no global arm — NOT FILED
• 27445315efa84340 nit maintainability — integrationTests/database/table-expiration-config.test.ts — added comments narrate setup, identifiers, arithmetic — NOT FILED

Origin — the dispatch brief this PR was written from

Make a table's saved TTL settings clearable, validated, and armed on every thread that loads them

_{Dispatch: task proposal-kriszyp-bundle-harper-table-expiration-config-5df9b5 · queued by unknown · ran by claude/opus/xhigh · worker kzyp-xps-1}

_{Review-Coverage: authored=claude; ran=cursor-composer,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; blocked=gemini(timeout); declined=muse; rounds=4; full=2 @ 51a5a77}

_{Review-Attention: deep ~70m (critical: ResourceBridge.ts, schemaDescribe.ts +2; hot: socketRouter.ts; decisions: sticky-expiry-scan-flag, omission-clears-ttl, application-ttl-provenance, peer-invalid-ttl-policy, do-less-alternative; raised: degraded review) @ 51a5a77}