Repository navigation
Keep a replicated write with no expiry from taking its transaction's first expiry - #4391
Draft
DavidCockerill wants to merge 8 commits into
Draft
DavidCockerill wants to merge 8 commits into
DavidCockerill wants to merge 8 commits into
Conversation
…first expiry The source apply loop stages every write of a source transaction under its first event (or the transaction wrapper) as the context, and _writeUpdate resolved options.expiresAt ?? context.expiresAt. A write whose event carries no expiry therefore took the first event's, so a replica (or any table fed by a subscribing source) later hid and evicted a row the origin kept. A source-applied write (options.isNotification) now takes its expiry only from its own event, falling back to its @expiresAt field and the table default as before. Closes #4390 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #4390 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…test comment Refs #4390 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the no-expiry assertions An ordinary write with no expiry now reads only what it read before. Refs #4390 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #4390 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #4390 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he read-hiding test a wider window A null options.expiresAt collapsed to undefined through ?? before; keep that, since an audit-only commit would otherwise encode it as expiry 0. Refs #4390 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #4390 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
⊙ Problem
A transaction writes a row with an expiry, then a row without one. A node applying it from a source gives the second row the first row's expiry. A replica applies every origin transaction this way. After that expiry the replica hides the row, and its cleanup scan can delete it. The origin keeps it. Present in 5.3.6 and 5.4.0.
Closes #4390
💡 Solution
@expiresAtfield or its table'sexpirationapplies, as for a standalone write.context.expiresAtwhen they pass none.expiresAt.🤔 Should this also repair rows already stamped with a borrowed expiry?
No. It changes new writes only. A replica row that already borrowed an expiry keeps it until rewritten. A row already evicted needs a resync from the origin. Nothing on the row distinguishes a borrowed expiry from its own.
⚖️ Alternatives
-1("no expiry") from harper-pro's receiver — skips the receiving table's ownexpirationand@expiresAt; custom sources stay broken.expiresAtkey in the options override the context — changes_writeUpdate's contract for every caller, not only writes applied from sources.Left out: two related gaps that predate this PR
-1is not replicated. An origin write withexpiresAt: -1stores no expiry, so a replica applies its own table default. Fixing it needs an audit-format change.main.🤔 Should a
transactionevent's ownexpiresAtcover the writes in it?No: each write uses only its own
expiresAt. A custom source that set one only on the wrapper no longer gets it on those writes; they fall back to their@expiresAtfield or tableexpiration. Nothing documented the old behavior, and restoring it is one condition.🔧 Changes
Product and architecture tour: which expiry a source-applied write stores, and where the borrowed one came from (one diagram)
Product and architecture tour
Which expiry a source-applied write stores
Which expiry does a write applied from a source or a replication peer get when its event carries none?
expiresAtin a source transaction took the expiry of the first write, or of thetransactionevent.@expiresAtfield, then its table'sexpiration, then none.Where the borrowed expiry came from
The apply loop reuses a transaction's first event as the context for every write in it. Expiry was the only per-write field that fell back to that context.
flowchart LR E1[First event, expiry T] -->|becomes| C[Transaction context] E2[Later event, no expiry] --> W[_writeUpdate] C --> W W -->|before: falls back to T| H[Row hidden after T] W -->|after: field or table default| K[Row kept]Every other per-write field (
nodeId,version,residencyId) is read from the write's own options only, so nothing else leaks the same way.isNotificationis set only by the apply loop'swriteUpdate, which also passes each event's ownexpiresAt. A-1still means "never expires", and a write's own deadline still wins._writeUpdateTable.ts:6834
Files
resources/Table.ts—_writeUpdate's expiry resolution: reads the write's own expiry first, and a source-applied write skips the shared context's.unitTests/resources/sourceTxnExpiry.test.js— the regression suite (see Verification).docs/reference/resources/resource-api.md— a short paragraph on a subscribe event'sexpiresAt.✅ Verification
sourceTxnExpiry.test.jsfails on base in 4 of 6 cases, on RocksDB and LMDB. All 6 pass with the fix.Plain; with this branch's head it keeps it.test:unit:mainis left to CI.Table.tsreworded, not re-reviewed._writeUpdateruns on every write. A write with no expiry in its options or context reads what it read before; only one that would take the context's expiry readsoptions.isNotificationtoo. No allocation, lock or await.Test cases
expiration: no stored expiry, still readable after the first expiresexpiration: 3600: gets the table default; a-1write stays without expiry@expiresAtfield: each keeps its field's deadlinetransactionevent with its ownexpiresAt: the write inside gets noneexpiresAtstill gets itGate results
Full unit gates ran at e990466 and the integration gate at fddf5ea, on harper-pro
main43599092 for the cluster run (base core 4f20ed5). Lint, format, design-doc and unit-partition checks pass.test:unit:maincannot finish in one process on this Mac. Later commits change only the form of the expiry expression and comments; the focused suites (sourceTxnExpiry,sourceTxnStreams,outOfOrderWriteExpiry,expirationWarning,caching,replicatedApplyFailure) were rerun on both engines at 16acead.test:unit:resources(RocksDB)rocksOpenConfigwriteBufferManagerFlushPolicy fails the same on base;originClosedFloorpasses alonetest:unit:resources(LMDB)rocksOpenConfigcase;historyReader"stops search windows at the lower retained-log boundary" fails on base too (2 of 2 runs)test:integration:allA related warning with a different cause
A 5.4.0-beta.2 replica also logged the "per-record expiresAt was set" warning for tables that do have an
expiration. That is a separate defect: a table a node received throughadd_nodelost its expiration, fixed for 5.4.0 by HarperFast/harper-pro#1233.🤖 Generated by Claude Code (Anthropic Claude Opus); posted via @DavidCockerill.
Release target: v5.4 — the issue's milestone; the code is also on v5.3, but no customer needs a v5.3 patch
Related PRs: #2155 overlaps (edits the same expiry line in
_writeUpdate; its version keeps the context fallback)Complexity: complicated
Deferred findings
378124750da8e5edmajor data-integrity —resources/RecordEncoder.ts— -1 never-expire override is not encoded in record metadata or audit, so replicas apply their table default — NOT FILED9f628a1ecdd9c2b6minor data-integrity —resources/Table.ts— a write sharing a source transaction's context without isNotification (derived or side-table write) still inherits the first event's expires — NOT FILEDReview-Coverage: authored=claude; ran=gemini,codex,gemini-carbon; late=gemini-carbon:adjudicated; adjudicated=domain; declined=cursor-grok,cursor-composer,muse; rounds=7; full=3 @ 57b9017
Review-Attention: study ~11m (critical: Table.ts; decisions: wrapper-expiry-semantics, gate-layer, suppress-at-writeupdate, perf-cost) @ 57b9017