Repository navigation
Skip relayed duplicates of a record's committed head without reading the transaction log - #3476
Conversation
Release cherry-pick
|
There was a problem hiding this comment.
Code Review
This pull request updates the Harper design documentation and implementation to improve the handling of replicated duplicates. Specifically, it introduces a priorWritesSkipped helper function in Table.ts to ensure that replicated writes are correctly identified and processed, and adds comprehensive test coverage in relayedHeadTieDuplicate.test.js to verify these scenarios. There are no review comments to address for this pull request.
…the transaction log A replicated write that matches the committed record's origin, version and log key (no explicit refs) is already applied. That was only recognized on a retry; on a first delivery the keyed dedup looked for the head's log entry, and when the receiver had none (a base-copied record) the exact-start lookup scanned to end-of-log and the write was applied and logged again, so every peer was re-sent it (harper#3467). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e skip A publish or relocate earlier in the same source transaction restamps the record at the transaction's version without staging it, so the put after it tied with that restamp and was dropped. The skip now requires every earlier write to the key in the transaction to have been skipped, which still lets a re-delivered multi-write transaction skip all of its writes from the record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…entation on it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ab34002 to
b81bc92
Compare
v5.3 has a single log key, so the head-tie skip keeps txnLogKey === txnTime and adds priorWritesSkipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v5.3 has a single log key, so the head-tie skip keeps txnLogKey === txnTime and adds priorWritesSkipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hout reading the transaction log (conflicts → v5.3) (#3478) * Skip relayed duplicates of a record's committed head without reading the transaction log A replicated write that matches the committed record's origin, version and log key (no explicit refs) is already applied. That was only recognized on a retry; on a first delivery the keyed dedup looked for the head's log entry, and when the receiver had none (a base-copied record) the exact-start lookup scanned to end-of-log and the write was applied and logged again, so every peer was re-sent it (harper#3467). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep a put that follows an unstaged same-key write out of the head-tie skip A publish or relocate earlier in the same source transaction restamps the record at the transaction's version without staging it, so the put after it tied with that restamp and was dropped. The skip now requires every earlier write to the key in the transaction to have been skipped, which still lets a re-delivered multi-write transaction skip all of its writes from the record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cover a put after a same-key relocate, and keep removalBefore's documentation on it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Name the relocate case in the head-tie design note Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep only the constraint in the head-tie comment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Resolve the v5.3 cherry-pick of #3476 v5.3 has a single log key, so the head-tie skip keeps txnLogKey === txnTime and adds priorWritesSkipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep a replicated put that shares its transaction with a same-key publish or relocate from being skipped as a duplicate (#3499) Port of 5d94a9a to v5.3. The head-tie identity now comes from the committed record whenever another write to the key already saved unskipped, and a source-applied relocate that ties with the committed head is skipped. v5.3 has a single txnLogKey (no origin-closed floor), so main's receivedLogKey/appendedLogKey checks map to `txnLogKey === txnTime`; the relocate keeps v5.3's plain return on an older version (#3183's skipped marking is not on v5.3), and the ref check keeps v5.3's form without main's priorStaged guard. (cherry picked from commit 5d94a9a) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SbvHaBBdQ7Cp7yVDuSD5GS Dispatch-Task: cherry-resolve-kriszyp_harper_3476-123ff057 --------- Co-authored-by: Kris Zyp <kriszyp@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
⊙ Problem
Replication receivers spend seconds per write on duplicates they already hold (#3467). The check looks up the write in the origin's transaction log. A base-copied record has no log entry, so on RocksDB the lookup reads to the end of the log. The duplicate is then logged again and re-sent to every peer, so an old stretch replays indefinitely.
🤔 Does this belong beside #3453?
Yes. #3453 hardens the deeper retry-path dedup. This PR covers first deliveries, which #3453 does not reach. Both edit the same design-note bullet, so whichever merges second keeps both guarantees.
💡 Solution
⚖️ Alternatives
🤔 Should a base-copied record count as logged under its version?
I chose yes. A copy drops the record's head ref, so a distinct write from the same origin, whose version equals the copied write's log key, would be skipped. That needs a source-reported version equal to a commit timestamp. Carrying the head's log key in copy frames would close it.
🔧 Changes
resources/Table.ts: the head-identity skip incommitdrops the retry-only condition and requirespriorWritesSkipped.resources/DESIGN.md: the existing note covers first deliveries and the publish/relocate exception.unitTests/resources/relayedHeadTieDuplicate.test.js: five cases through the replication subscription path.✅ Verification
priorWritesSkippedis removed.test:unit:resources: 4630 passing. Two failures (evictionBlobsresequenced-version case,replicatedDeleteRedelivery[put, delete] case) fail the same way on unmodifiedmain.🤖 Generated by Claude Code (Claude Opus 5.5); posted via @kriszyp.
Release target: v5.3 — fixes code from #3366, a backport to v5.3; the affected cluster runs 5.3
Related PRs: #3453 overlaps (same design-note bullet), #3375 overlaps (changes relocate, which priorWritesSkipped relies on), 6 others independent
Complexity: complicated
Review-Coverage: authored=claude; ran=gemini,cursor-composer,codex,cursor-grok; adjudicated=domain; declined=muse; rounds=5; full=2 @ ab34002
Review-Attention: study ~15m (critical: Table.ts; decisions: head-tie-on-first-delivery, skip-without-append, copy-row-log-identity, sibling-write-types; raised: degraded review) @ ab34002