Skip to content

Skip the transaction-log probe for a replicated publish keyed above everything its origin's log holds #4341

Description

@ldt1996

Since #4339, every source-applied publish on RocksDB probes its origin's transaction log at the received key (query({ start, exactStart: true })) before it is applied, so a re-sent message that this node already logged is skipped instead of delivered again. A first delivery always misses. It costs about 2.3 µs per publish on a sequential apply and 0.4 µs with concurrent applies (60k publishes, measured on the PR).

A message whose key is above every key in the origin's log on this node cannot have been logged, so a first delivery in commit order could skip the probe. Nothing cheap answers that today:

  • the record's version cannot, because a late publish lowers it;
  • rocksdb-js tracks each segment's largest key for purge (appendMaxExact, scanMaxEntryTimestamp), but does not expose a log's largest committed key.

Proposal. A rocksdb-js read of a log's largest committed key that is cheap enough to call per write, or a per-log high-water mark core keeps in a shared buffer and restores from that read at open. isMessageLogged would then probe only a key at or below it.

Check. The PR's sequential A/B (60k source publishes), base against fix, shows no difference for in-order first deliveries.

🤖 Generated by Claude Opus 5.5 (Claude Code); posted via @ldt1996.

Activity

  1. added theissue type on Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    P3

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions