Repository navigation
Description
Problem
A durable MQTT session that reconnects to the node that saved it resumes from its single saved position (the scalar startTime). A transaction that lands after that save with a log key below the saved position is never replayed, so the subscriber silently misses it. Nothing logs or reports the skip.
The remaining source is another origin: a remote node's transaction replicates in after the save but carries that origin's earlier timestamp. Replication lag is routinely seconds, so this is not rare. This is the cross-origin half of A resumed subscription can skip a transaction that commits after its cursor with a lower key (harper#2928).
Local transactions are already covered on main: Hold a durable subscription's position below every local write still in flight (harper#3635) holds the saved position below the lowest outstanding local reservation.
Every shipped release behaves this way. Durable MQTT sessions no longer resend acknowledged messages or freeze their position once the storage engine issues checkpoints (harper#3778) keeps it on purpose: on the saving node and database generation, the scalar is authoritative and the checkpoint token is used only for another node or a restored copy. server/DESIGN.md lists the gap under "Not covered".
The guarantee already exists per origin
The checkpoint token already bounds both sources. Each origin's certified closed floor F_O means no transaction from origin O can still commit with a key below F_O:
liveBoundaries(resources/Table.ts, insubscribe) sets each origin's boundary toF_O, lowered by events still queued or held for that origin.encodeCheckpoint(resources/subscriptionCheckpoint.ts) stores the boundaries astime - overlapplusexceptionsfor origins that lag further.MAX_OVERLAP_MSis 60 s.
So the token alone never misses a late commit. Resuming from the token alone, as main did before harper#3778, failed for three other reasons, all fixed or avoided below.
Proposed resume rule (same node, same database generation)
- Per-origin replay start: for each remote origin, start at
min(savedPosition, boundary_O), taken from the latest token. The local origin keeps the saved position, which harper#3635 already bounds. WhenF_Ohas already passed the saved position, that origin replays nothing. - Do not lower the saved position. It stays the session's reported position. On
mainbefore harper#3778, the session capped it below the token (scalarCap/lowerScalar), which froze a quiet topic's position (integrationTests/mqtt/durable-resume.test.ts:505). - Never before the topic's SUBSCRIBE. A topic's first token can predate its subscription (
durable-resume.test.ts:722). - Suppress acknowledged redelivery with the QoS 1 and QoS 2 receipts harper#3778 adds (
durable-resume.test.ts:483). - Pruned window: if history between
boundary_Oand the saved position is gone, resume from the saved position (today's behavior) rather than resetting the session.
With harper#3635, this closes harper#2928 for same-node resume.
Cost
- Each same-node reconnect replays, per topic, the remote-origin floor lag: about 30 s, capped at 60 s.
- QoS 0 subscribers receive duplicates from that window. QoS 1 and 2 duplicates are suppressed by receipts up to the 1,000-receipt cap; past the cap, the existing documented resend applies.
- Unmeasured: replay cost on high fan-out QoS 1 topics. A benchmark should gate the change.
Acceptance
- A same-node resume delivers a late commit from a remote origin that lands below the saved position after the save.
integrationTests/mqtt/durable-resume.test.tsstill passes on a rocksdb-js build withsyncWal, plus the pinned version.- Unit tests in
unitTests/server/durableSessionCheckpoint.test.jsfor the remote late commit and for the pruned-window fallback. server/DESIGN.mdmoves the same-node late-commit gap out of "Not covered".
Depends on harper#3778 (receipts and the generation-bound scalar). Part of Add per-origin closed timestamp floors for compact, cross-node subscription resume (harper-pro#922).
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity
[-]Same-node durable MQTT resume skips late commits below the saved position; replay from per-origin floors[/-][+]Same-node durable MQTT resume skips late commits from other origins below the saved position; replay from per-origin floors[/+]