Skip to content

Replication W7: Robust sharding (residency-vs-routing split) #436

Open
18 of 83 issues completed
Open

Description

@kriszyp

Workstream W7 of #430 · robust sharding (the residency-vs-routing split)

Summary

Sharding is built on "residency" — a node list computed per write and interned to a persistent residencyId. The core flaw is that residency is overloaded: a transient per-request replicateTo is laundered into the same persistent residencyId as a durable shard policy, so request-scoped intent mutates placement and triggers invalidation broadcasts to nodes that were never targets. Compounding gaps make sharding crash-prone. (Most of this is core Table.ts work with replication implications.)

Root cause / current state

Design direction

  1. First-class distinction between durable residency (shard policy) and per-request targeting, so replicateTo never re-interns a persistent residencyId and never broadcasts invalidations to never-target nodes.
  2. One uniform replicateFrom/locality choke point across REST/SQL/blob/source-load — not per-path guards.
  3. Read paths are placement-neutral — a non-owning read returns local/empty (or fetches without re-persisting residency) and never rewrites ownership.
  4. Residency-function contract — invoked only with the fields it declares it needs (or always the full record); config-time validation that a sharded table has a live function and a populated server.shards.
  5. Merge-aware scatter-gather (limit/offset after the cross-shard merge); exhaustive constraint-error capture on relocate/apply; self-exclusion at residency computation.

Scope

  • Separate durable residency from per-request replicateTo (no persistent re-intern)
  • Single uniform replicateFrom/locality choke point (REST/SQL/blob/source-load)
  • Placement-neutral reads (no ownership rewrite on read/cache-load)
  • Residency-function field-presence contract + config-time validation
  • Merge-aware limit/offset for scatter-gather
  • Exhaustive constraint-error capture; self-exclusion at routing
  • Expose shard count (Expose shard count via cluster status API #204) → feeds W8

Retires / advances

Dependencies

Largely independent (core-side). Benefits from W8 for the shard-count exposure.

Effort / risk

L / medium. The residency-vs-routing split is the core refactor.

Acceptance criteria

  • A replicateTo update neither deletes nor leaks on non-target nodes.
  • X-Replicate-From: none is honored on blob and SQL paths.
  • A sharded write lands on exactly the residency nodes; a SELECT on a non-owner doesn't cache.
  • limit returns the expected count across shards; sharding misconfig surfaces as an error, not silent full replication.

🤖 Filed by Claude on behalf of Kris.

Activity

  1. added
    enhancementNew feature or request
    area:replicationReplication, cluster sync, peer connections
    area:clusteringClustering, sharding, replication topology
    on Jun 20, 2026
  2. added this to the v5.3 milestone on Jun 20, 2026
  3. added theissue type on Aug 7, 2026
  4. modified the milestones: v5.3, v5.4 on Oct 10, 2026
  5. self-assigned this
    on Oct 20, 2026
  6. 69 remaining items

  7. added sub-issues on Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:clusteringClustering, sharding, replication topologyarea:replicationReplication, cluster sync, peer connectionsenhancementNew feature or request

Fields

Priority

P2

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions