Skip to content

The @expiresAt sweep never evicts a record whose field is a date string #3765

Open

Description

@kriszyp

Problem

A record whose @expiresAt field holds a date string (for example an ISO-8601 timestamp) is never removed by the @expiresAt sweep (runRecordExpirationEviction in resources/Table.ts).

  • The field's index key is the string itself. The sweep scans index.getRange({ start: true, end: Date.now() }), and string keys sort after every number, so the scan never reaches them.
  • For a row stored without a deadline in its metadata (written before the field was stamped into it), the sweep's fallback compares the raw field with < Date.now(), which is false for a string.

Reads are not affected: the write path parses the string (fieldExpiresAtMs) into the stored deadline, so the record is read-hidden once it passes. It stays on disk until a read evicts it or the metadata cleanup scan runs; that scan is armed best-effort on write and is not re-armed after a restart with no writes.

A related case in the same fallback: a row stored with no deadline in its metadata but a past @expiresAt field (written with an explicit expiresAt: -1, or a source fill, which derives expiry from the source rather than the field) is evicted by the sweep's field comparison, while reads treat it as live.

Repro (verified on both engines)

const T = table({ table: 'IsoProbe', database: 'test', attributes: [
  { name: 'id', isPrimaryKey: true }, { name: 'deadline', expiresAt: true, indexed: true }] });
await T.put('iso', { id: 'iso', deadline: new Date(Date.now() - 6000).toISOString() });
await T.put('num', { id: 'num', deadline: Date.now() - 6000 });
// run the sweep (RocksDB also needs harper#1481's getValues)
// 'num' is evicted; 'iso' remains, with a past stored expiresAt, and get('iso') returns null

Fix direction

Index (or sweep) the parsed deadline rather than the raw field value, and use fieldExpiresAtMs in the no-metadata fallback. Overlaps Fix RocksDB @expiresAt eviction sweeps, which rewrites this sweep.

Found while fixing out-of-order writes that replaced a newer record's expiry (PR to follow on branch fix/out-of-order-write-keeps-head-expiry).

Review-Deferral: f4a4b633e56e3cea severity=minor category=correctness

🤖 Generated by Claude Opus 5.5 (Claude Code dev-agent); posted via @kriszyp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    P3

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions