Repository navigation
Description
Problem
replicatedConfirmation: N is meant to tell the writer that N other nodes now hold the write. Today it only tells the writer that N peers have advanced past the write's transaction time for that database. On a table with a record-based residency list, those are different things:
- A non-resident peer receives only an invalidation entry for the record (an INVALIDATED stub), not the record, yet its per-database applied watermark passes the write's
txnTimeand it is counted. - A peer excluded by
replicateToreceives nothing for the record, yet any later write or sequence update for the database advances its watermark pasttxnTime, and it is counted.
So a write can resolve "confirmed by N" while zero other nodes hold a complete copy. The caller gets no signal; the promise silently means less than it says.
Where it happens: replication/knownNodes.ts — createConfirmationWaiter counts any peer (by identity, harper-pro#633) whose CONFIRMATION_STATUS_POSITION for the database is >= txnTime; the replicationConfirmation(...) registration bounds the count only by the database's recipient set (getReplicationRecipients), not by the record's residency.
Context: documentation#723 was about to document this as the contract. It is on hold until this is fixed.
Required behavior
A confirmation counts only peers that hold a valid (complete) copy of the written record(s). Tables without residency lists (every peer resident) behave exactly as today.
Proposed shape
- Core computes, per write, the set of resident peers it already knows when it applies residency (record-based residency or
setResidencyById), and passes it into thereplicationConfirmationcallback alongsidetxnTimeand the count. createConfirmationWaiterignoresonConfirm(and thealreadyConfirmedPeersseed) for any peer outside that set.- No protocol change: eligibility is decided on the writer, and the existing watermark ack is reused, so mixed-version receivers need nothing new.
- The writer itself does not count toward N (it stays resident per harper#3128).
Open questions (recommendations from triage)
- Transactions spanning records with different residency lists. Recommend: require N confirmations per record from that record's residents (equivalently, a peer counts for a record only if resident for it), rather than one intersected set.
- N larger than the number of other resident peers. Recommend: fail fast with a clear error at write time. Silently capping N reproduces this bug; timing out (today's behavior) costs the full confirmation timeout for a request that can never succeed.
Acceptance
- A test with a record-based residency list where the only non-writer resident is down: a write with
replicatedConfirmation: 1does not confirm from a non-resident orreplicateTo-excluded peer, and times out or errors per the ruling above. Fails on current main. - Unsharded tables: existing confirmation tests unchanged.
- A multi-record transaction with differing residency covers question 1's rule.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity
Name the feasibility check's database from the first write still in t…
Bump core for the feasibility check database name
Name the residency lists from the narrowed groups in the waiter's tim…
Check confirmation feasibility only before the first submission, and …
Give every residency test request a deadline, and bump core for the f…
Key the pre-submission confirmation check and its lists to the native…
Bump core for the handle-keyed confirmation check
Count only a record's residents toward replicatedConfirmation, and re…
Count only resident peers toward replicatedConfirmation (#1032)
kriszyp commentedon Oct 13, 2026
Fixed by #1032 (Count only resident peers toward replicatedConfirmation), merged 2026-10-11. Closing; the merged PR referenced this issue without a closing keyword.
— Claude Opus 5.5 (triage pass)