[kriszyp]

⊙ Problem

Table.publish is documented to leave the record unchanged, but it rewrote it from a stale copy. A put, patch or create earlier in the same transaction was undone in the stored row while the index kept the new value. A standalone publish cleared the record's expiration, residency id and invalidated-stub flag.

🤔 Is this the whole fix?

No. An older message still rewinds the record's version (#3637): keeping it would drop the message from RocksDB per-record replay.

💡 Solution

• A publish rewrites the record as earlier writes in its transaction left it.
• It keeps the record's expiration, residency id and local-only, invalidated and replica flags.
• The message keeps its own residency for routing; the record keeps its own.
• A re-delivered replicated patch after a publish still applies once.

🤔 Accept that [put K, publish K] in one transaction replays the put, not the message?

For now: RocksDB replay finds the first entry at their shared log key. Fix: #3661.

🤔 Accept publish K; relocate K; patch K re-storing the full body on an excluded host?

For now: a relocate stages nothing, so the patch bases on the publish's full record. Extends a known gap (DESIGN.md).

⚖️ Alternatives

• Audit-only publish — messages would drop out of per-record replay.
• Fix only the metadata — leaves the index inconsistent.

🤔 Should Cache-Control: max-age on a publish still set expiration?

Main applied it to the record on caching tables; now the record keeps its own expiration.

🔧 Changes

• resources/Table.ts _writePublish: joins the same-key write chain, takes its base from priorStagedWrite or the stored entry, keeps the record's residency and expiration, and stages that base marked unchanged with the removal identity a later delete would otherwise find. Look hardest here.
• resources/Table.ts _writeUpdate/_writeInvalidate: a publish-staged base is no earlier change for the tie-break, the re-delivery checks and the invalidation base-copy skip; on unaudited tables a restamp still counts as program order. Update, invalidate and delete stage the metadata a following publish carries. The followsAppliedWrite and identityBase comments now say which writes restamp.
• resources/RecordEncoder.ts: recordResidencyId sets the record's residency separately from the audit entry's.
• resources/DatabaseTransaction.ts: the unchanged field on stagedEntry.
• resources/DESIGN.md: the publish note in the same-key chain section, publish among the chained writes, and the delete re-delivery note.

✅ Verification

• publishRecordState.test.js on both engines covers same-transaction writes, metadata, local-only flags, residency routing, re-delivery and delete copies; 9 of 14 cases (RocksDB) and 11 of 14 (LMDB) fail on main.
• Updated expectations: relayedHeadTieDuplicate, subscriptionBufferedState and sourceFillConditionalWrite.
• test:unit:resources, both engines: only timing tests (pass alone) and old-binding full-text checks fail.

PR QA: Do concurrent publishes and writes to the same keys across HTTP workers leave primary rows, indexes and expirations consistent? — queued @ 1b3b892, qa-pr-harper-3671-1b3b892-r2

Publish cost measurements

Single-transaction Table.publish to a warm key, µs per publish; load average 7–30, turbo on.

record	main	this PR
~300 B	42–117	54–99
64 KB	143–179	217–244

Uncached getEntry: 3.3–14 µs vs cached 0.4–1.6 µs (small record), 29–35 µs at 64 KB.

Docs: none; publish now matches them.

_{🤖 Generated by Anthropic Claude (Opus 5.5, dispatch dev agent); posted via @kriszyp.}

Release target: v5.4 — fixes behavior on main; no reported need on an older line
Related PRs: #3453 overlaps (re-delivery dedup checks in _writeUpdate), #3506 overlaps (the unaudited tie rule), #3529 overlaps (changes its source-publish fill expectation), #3527 overlaps (publish carries the shard slot and complete record), 17 others independent
Complexity: complicated
Framing-Verdict: chosen-approach-sound (019fa91eec79)

Deferred findings

• f2887ce6cd9751ed major correctness — resources/Table.ts — older/tied/chained publish sets VERSION_REUSED on the unchanged record, hiding it from full-text results — filed A publish that does not advance the version hides an unchanged record from full-text results #3660
• 3d73554ed565a886 major performance — resources/Table.ts — keyed publish adds an uncached commit-base reload and decode per message — filed A static publish re-reads its record uncached; large retained records pay ~50% per message #3665
• 2b991d7842eb5ffa major data-integrity — resources/Table.ts — older publish rewinds version so a chained put/delete overwrites a newer record — filed An out-of-order publish rewinds the record's version, letting a stale delete or write through #3637
• b49828595b981a48 minor data-integrity — resources/Table.ts — [publish K, put/patch K] in one transaction: replay omits the later non-message write sharing the log key — filed Per-record replay delivers only the first of a transaction's writes to one record #3661
• c9fcb19803b01b6e minor correctness — resources/Table.ts — LMDB fill yields to an unchanged publish and leaves the stub invalidated while RocksDB completes it — filed On LMDB a source fill yields to the source's own publish and never caches the key #3662
• 29a42bdfed78357c minor correctness — unitTests/resources/sourceFillConditionalWrite.test.js — LMDB fill yields to an unchanged publish and leaves the stub invalidated while RocksDB completes it — NOT FILED
• c7d6a0469e0756b7 nit correctness — resources/Table.ts — RocksDB skips a chained invalidate that ties with a publish restamp; LMDB applies it — filed A replicated [publish K, invalidate K] skips the invalidation on RocksDB but applies it on LMDB #3666

🤔 defer this PR's major correctness finding f2887ce6cd9751ed (older/tied/chained publish sets VERSION_REUSED on the unchanged record, hiding it from full-text res) to #3660 instead of fixing it here?

🤔 defer this PR's major performance finding 3d73554ed565a886 (keyed publish adds an uncached commit-base reload and decode per message) to #3665 instead of fixing it here?

Late Grok findings

2 surviving (highest major) — Grok reviewed 71e2425

• major — resources/Table.ts:8295 — a put/patch/delete/invalidate followed by a publish to the same key in one transaction gives both audit entries the same log key (appendedLogKey === txnTime, so no additionalAuditRefs). On RocksDB, resolveAuditHead (resources/Table.ts:1707) then calls auditStore.getSync(txnTime, …), and if
• major — resources/Table.ts:8307 — an older publish, or a same-version tie (including the RocksDB same-transaction put→publish case, where the reloaded base's version equals txnTime), writes newVersion <= existingEntry.version. That sets VERSION_REUSED (resources/RecordEncoder.ts:1002-1003) on a record whose value did not

Not standing: none. Grok ran 14m (14m from launch to report); the fold adjudication (cursor-grok@71e2425b88ea-563a813f, claude) took 2m.

Late Grok findings

7 surviving (highest major) — Grok reviewed 71e2425

• major — resources/Table.ts:8287 — A put followed by a publish on the same record in one transaction writes both audit entries under the same log key. On RocksDB, getSync returns the first entry matching the record at that key (resources/RocksTransactionLogStore.ts:222-225), which is now the put, because the publish chains af
• major — resources/Table.ts:8307 — An older publish rewrites the record at existingEntry.version, which satisfies versionIsReused (resources/RecordEncoder.ts:157-158) and sets VERSION_REUSED (resources/RecordEncoder.ts:1002-1003) on a record whose value did not change. isFullTextSearchEntryCurrent (`resources/Table.ts:194
• major — resources/Table.ts:1741 — collectMessageBranches reads only the direct refs and never follows previousAdditionalAuditRefs. The encoder drops middle log-key refs when they exceed its byte budget (resources/RecordEncoder.ts:416-419, :463). A message whose only remaining link is a later message's `previousAdditionalAu
• major — resources/Table.ts:8261 — Every keyed publish now takes a commit-base read transaction and reloads the base entry at commit (baseReadTxn, reloadCommitBase: keyed). This is extra read I/O on every publish to a record, and the PR includes no measurement of it.
• major — resources/Table.ts:7940 — Each RocksDB subscribe with a start time now allocates a Map, builds one string key per replayed entry (:7967), and does one log getSync per non-head ref (:1753). The PR includes no measurement of this replay path.
• major — resources/Table.ts:8330 — The publish stagedEntry does not carry the removal identity. With delete → publish → delete on the same record and identity in one transaction, the second delete is logged again. Severity is a nit on the merits; it is graded major only because of the unmeasured hot-path floor. Follow-up issue alre
• nit — resources/Table.ts:8254 — Several added comments restate the code: resources/Table.ts:1739, :8254, :8263, :8288, and the test-description comments in unitTests/resources/publishRecordState.test.js and unitTests/resources/subscriptionBufferedState.test.js. One existing comment is reported stale near `resources/Tab

Not standing: 2 separate-issue. Grok ran 13m (13m from launch to report); the fold adjudication (cursor-grok@71e2425b88ea-8398ace6, claude) took 2m.

Origin — the dispatch brief this PR was written from

Table.publish / record.publish (_writePublish in resources/Table.ts) is not a chained write and rewrites metadata. It sets entry: this.#entry without chainsStagedState/reloadCommitBase/priorStagedWrite, passes flags & LOCAL_ONLY only, expiresAt: context?.expiresAt, residencyId: options?.residencyId and newVersion = txnTime, and updateRecord's message path drops HAS_EXPIRATION, residency id and INVALIDATED. Effects: same-transaction put/patch then publish writes a stale base back (primary/index divergence); deferred writes before a publish are dropped; a standalone publish wipes expiresAt, residency id and the INVALIDATED flag (promoting a stub to a full-looking record); an out-of-order publish rewinds the record version so a stale delete passes. Make it chain like _writeInvalidate (#3322): chainsStagedState and reloadCommitBase, base from priorStagedWrite(write)?.stagedEntry ?? existingEntry, publish stagedEntry unchanged, deferred create/update/put applied first; carry the existing expiresAt, residencyId and flags & (LOCAL_ONLY|INVALIDATED|REPLICA); newVersion = Math.max(txnTime, existing version) while the message audit entry keeps the txn timestamp; no updateIndices. Update the resources/DESIGN.md note that publish was left unchained.

Acceptance

Tests on rocksdb and lmdb: put(v2) then publish in one transaction leaves primary store and index on v2; standalone publish on a record with expiration, one with a residency id, and an INVALIDATED stub leaves each unchanged; publish at t1 < t2 keeps version t2 and a stale delete at tMid is rejected. Show each new test failing on origin/main sources (forced rebuild) before the fix.

_{Dispatch: task harper-writepublish-chain-preserve · queued by finding-triage/wave-40 · ran by claude/opus/high · worker kzyp-xps-1}

Late Grok findings

3 surviving (highest major) — Grok reviewed 1b3b892

• major — unitTests/resources/sourceFillConditionalWrite.test.js:284 — On RocksDB, a fill commit after a same-transaction publish probably throws the fill away. If so, this assertion fails on the default-engine (RocksDB) CI shard, and resources/DESIGN.md:532 documents an engine split that does not exist. Disputed and not executed. Scope: in-scope — Convergence-Class:
• minor — resources/Table.ts:5600 — A replicated transaction publish K; relocate K; patch K stores the full record on a host the residency policy excludes. Documented at resources/DESIGN.md:528 but not fixed. Scope: in-scope — Convergence-Class: original-change
• nit — resources/Table.ts:9498 — New publish comments partly repeat the publish section of resources/DESIGN.md. Scope: in-scope — Convergence-Class: previously-adjudicated

Not standing: 1 merged. Grok ran 8m (8m from launch to report); the fold adjudication (cursor-grok@1b3b892f5700-4c3fd75e, claude) took 1m.

_{Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; declined=muse; rounds=11; full=3 @ 1b3b892}

_{Review-Attention: deep ~25m (critical: DatabaseTransaction.ts, RecordEncoder.ts +1; decisions: publish-ignores-context-expiresat, keyed-publish-joins-chain, document-relocate-gap-not-fix, engine-fork-to-issue, sparse-staged-entry, do-less-alternative, deferred-f2887ce6, deferred-3d73554e) @ 1b3b892}