[kriszyp]

⊙ Problem

Older writes (out-of-order replication, or an explicit older timestamp) merged fields under newer fields but replaced the record's expiry with their own. A stale past expiresAt hid a live record; the next read evicted it. A write with no expiresAt stripped the expiry or cut it to the table default. The @expiresAt sweep also evicted on the raw field when the stored expiresAt kept the record live.

🤔 Is the requirement right?

Yes: expiry follows the fields' last-writer-wins rule unless the merge changed the @expiresAt field. Math.max would give a record with no expiry the older write's.

💡 Solution

• An older write that leaves the merged @expiresAt field unchanged keeps the expiry of the record it merged onto, exactly, including none.
• Its audit entry still logs its own expiry (new auditExpiresAt option), because replicas and crash replay apply it.
• The sweep decides with hasExpired(entry) when a deadline is stored.

Refs #3575.

⚖️ Alternatives

🤔 Which record is "the head" inside a transaction (look hardest here)

The record the write merged onto: an earlier same-key write's staged entry, else the stored one. existingEntry differs by engine. So a second older write in one transaction keeps the first one's field renewal even with an explicit expiresAt, as separate deliveries would.

🤔 An older write that changes the @expiresAt field still decides the expiry

If the newer record's expiry came from an explicit expiresAt, an older write that sets the field still replaces it, so delivery order matters, as on main. Fixing it needs the record to store where its expiry came from: #3766.

🤔 The audit entry keeps the write's own expiry

Logging the head's expiry instead would give a relayed write a different expiry than its origin logged. A derived audit expiry still comes from the merged record; that predates this PR (#3766).

Other options considered

Option	Why not
Keep the stored expiry inside RecordEncoder when a version is reused	LMDB stores an older merge at its own older version, and the encoder cannot see whether the field changed
Rebuild expiry from the audit walk	The walk runs only on audited tables and can be capped; the stored expiry already is the newest write's
One expiresAt for both the record and the audit entry	Changes what replicas and replay apply for the older write
A per-write base grouped by version	Two extra stores on every write, and a grouping rule that needed three fixes; the merge base gives the same answers with none

🔧 Changes

• resources/Table.ts: the expiry decision for an older write and the auditExpiresAt it declares and passes to updateRecord; the staged expiresAt that a later write to the key in the transaction reads comes from main's #3671; and the sweep predicate.
• resources/RecordEncoder.ts: updateRecord takes an optional auditExpiresAt, which sets the audit entry's expiry (append path, and the replacement path) and its presence bit; without it, behavior is unchanged.
• resources/DESIGN.md: the expiry rule for merged writes, folded into the TTL entry so the file gains no lines, and a corrected note on what the sweep decides on.
• Tests: the new outOfOrderWriteExpiry.test.js. evictionBlobs.test.js and caching-rocks-database.test.js used to renew a record through an older write's expiresAt option, which no longer renews; they now renew through the @expiresAt field (or put the expiry on the newer write) and keep their assertions.

✅ Verification

• outOfOrderWriteExpiry.test.js: RocksDB 14 passing; LMDB 10 passing, 4 RocksDB-only skipped. On origin/main, 7 fail on RocksDB and 5 on LMDB.
• The staged QA test is included; its positive control renews through the field.
• Expiry and eviction integration tests (12 files): 63/63 passing.
• Rebased onto main dede2d3da; the only conflicts were the DESIGN.md bullet position and Table.ts's staged-entry literal, where main's #3671 already stages expiresAt. 13 related test files pass (RocksDB 299, LMDB 182).

Full suites, performance, and known gaps

• test:unit:resources, RocksDB: 5763 passing, 8 failing. All 8 pass when re-run alone (345/345); they are timeouts on a host at load average ~21.
• LMDB: 4365 passing, 1 failing (fullTextDerivedIndex, a wall-clock slice deadline with a fake engine and no table writes; it failed 2 of 4 runs on this host).
• After the final design, 49 related resource files: 965 passing on RocksDB and 624 on LMDB, 0 failing.
• test:integration:all: 3003 passing, 4 failing. All 4 are OllamaBackend tests that need a local Ollama.
• test:unit:main did not finish within 55 minutes on this host. The 4 component files that had failures before the cutoff pass 151/151 alone.
• Performance: 100k in-order puts, audited RocksDB table, perf stat, best of 3 under bench-run: origin/main 32.61 G vs branch 32.56 G user instructions (36.9 vs 35.7 µs/op CPU), within the 6% run-to-run spread. Unaudited tables not measured.
• Not covered: real multi-node replication, and crash replay of the merged writes.
• The @expiresAt sweep never reaches a field stored as a date string, and its field fallback still evicts a row stored with no expiry. Both predate this PR and are in The @expiresAt sweep never evicts a record whose field is a date string #3765. The sweep still cannot run on RocksDB (@expiresAt per-record TTL never evicts on RocksDB (default engine) — index.getValues() is LMDB-only #1481); the sweep tests stub getValues.

PR QA: Does a merged out-of-order write keep the newer record's expiry across a crash replay? — queued @ 4988975, qa-pr-harper-3774-4988975
Docs: none needed; the documented @expiresAt and expiresAt behavior is unchanged.

_{🤖 Generated by Claude Opus 5.5 (Claude Code dev-agent); posted via @kriszyp.}

Release target: v5.4 — fixes current behavior on main; no customer need for an older-line backport is known
Related PRs: #2155 overlaps (both change the @expiresAt sweep predicate), #3671 overlaps (merged; its staged expiresAt is kept), 3 others independent
Complexity: complicated

Deferred findings

• fb657eab6be8572e major data-integrity — resources/Table.ts — Older field-changing write can override a newer head's explicit expiry — NOT FILED
• 48b59b13c48d1032 minor data-integrity — resources/Table.ts — audit expiry derived from merged record's head @expiresAt field — filed An out-of-order write's audit entry logs the newer record's @expiresAt deadline as its own #3766
• f4a4b633e56e3cea minor correctness — resources/Table.ts — sweep fallback compares raw @expiresAt field now when no stored expiry, wider than fieldExpiresAtMs; evicts a row reads treat as live — filed The @expiresAt sweep never evicts a record whose field is a date string #3765
• 42f0460684266478 nit maintainability — resources/Table.ts — Existing expiry-precedence comment repeats the design note — NOT FILED

Late Grok findings

4 surviving (highest major) — Grok reviewed b60281d, ruled against 9c5f795

• major — resources/Table.ts:6812 — every in-order staged put/patch allocates a second { value, expiresAt } object, held on write.expiryBase until transaction cleanup (100k-row transaction → 100k extra retained objects). No measurement for this path; write.stagedEntry (resources/Table.ts:6801-6807) could carry expiresAt inst
• minor — resources/Table.ts:6812 — the base advances only when precedesExisting >= 0. Scenario: in one RocksDB multi-version frame, an older write changes the @expiresAt field and stores the precedence-resolved deadline. A later older write to the same key still compares against the earlier base, so it can restore the head's e
• minor — resources/Table.ts:6650 — a derived audit expiry is read from the merged record, so the audit entry can log the head's @expiresAt deadline instead of the write's own. resources/DESIGN.md:132 now documents this; that narrows the contract but does not change the behavior.
• nit — resources/Table.ts:12635 — the added comments restate nearby code. Sites: resources/Table.ts:12635, resources/DatabaseTransaction.ts:765, unitTests/resources/outOfOrderWriteExpiry.test.js:1-3, unitTests/resources/outOfOrderWriteExpiry.test.js:48, unitTests/resources/evictionBlobs.test.js:226-227. The comments on the

Not standing: 1 resolved, 1 merged, 1 separate-issue. Grok ran 27m (27m from launch to report); the fold adjudication (cursor-grok@b60281def34f-29b424a4, claude) took 1m.

Origin — the dispatch brief this PR was written from

(1) In Table._writeUpdate (resources/Table.ts ~5724 and ~6646) an older out-of-order write (precedesExisting < 0) sets expiresAt from options/context/table default, overwriting or stripping the newer head's existingEntry.expiresAt (read-hiding or evicting a live record). When precedesExisting < 0 and the merged @expiresAt field did not change, keep the head's stored expiry exactly (existingEntry.expiresAt, including 'none'); do not use Math.max, which would give a no-expiry head the older write's expiry. (2) runRecordExpirationEviction (~12618) decides expiry from the raw field value; use hasExpired(recordEntry) when recordEntry.expiresAt is defined, falling back to the field value. Do not attempt ISO-string @expiresAt eviction in this PR (unproven reachability; mention it as a follow-up if you confirm it).

Acceptance

Staged test passes 5/5 on rocksdb and lmdb; existing evictionBlobs and expiresAtAttribute suites stay green. Show each new test failing on origin/main sources (forced rebuild) before the fix.

_{Dispatch: task harper-out-of-order-write-keeps-head-expiry · queued by finding-triage/wave-43 · ran by claude/opus/xhigh · worker kzyp-xps-1}

Late Grok findings

2 surviving (highest major) — Grok reviewed 3465e45

• major — resources/Table.ts:6715 — An older out-of-order patch that adds the @expiresAt field and passes an explicit expiresAt: 0 stores expiry 0 over the newer head's live expiry. fieldExpiresAtMs returns a number for the merged field and NaN for the head that has no field, so the Object.is check is false and the keep-head
• minor — resources/Table.ts:12863 — When a row has no stored expiry metadata, the sweep falls back to value[expiresAtName] < Date.now(). That test is wider than fieldExpiresAtMs (resources/Table.ts:13022), which rejects negative values. Example: a put of {deadline: -1} stores no expiry (resources/Table.ts:6711), reads sti

Not standing: 2 separate-issue. Grok ran 19m (19m from launch to report); the fold adjudication (cursor-grok@3465e4506259-e50ecd09, claude) took 1m.

_{Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain,domain(codex); declined=muse; rounds=11; full=6 @ c8e957b}

_{Review-Attention: study ~13m (critical: RecordEncoder.ts, Table.ts; decisions: expiry-owner, do-less-alternative) @ c8e957b}

[gemini-code-assist]

Code Review

This pull request ensures that an out-of-order write merged under a newer head preserves the head's expiration time (TTL) rather than overwriting it with a default or stale value, while still logging its own resolved expiry in the audit entry. It updates Table.ts to track and apply this precedence, adjusts RecordEncoder.ts to handle the separate audit expiration metadata, and updates the @expiresAt sweep to respect the stored metadata expiry. Additionally, comprehensive unit tests have been added to verify this behavior across various scenarios, including replicated frames and eviction sweeps. There are no review comments to assess, and I have no feedback to provide.