Branched databases for application isolation #642

Summary

Allow a deployed application to be configured to use a branched copy of one or more databases. A branch is a writable, isolated fork of a base database, invisible to other applications and to replication. The app's code is unchanged — it imports databases.data from harper exactly as it does today, but the binding resolves to the branch.

A branch is durable and deterministically named: it lives at <storage>/`branches`/<app>/<db>, derived only from the application and base database names, and is adopted again on restart rather than re-checkpointed. Nothing process-local appears in the path. That is deliberate — an isolated application deployed onto a cluster is expected to be transparently available across it, so every node must resolve the same application's branch to the same place, and that identity is what a future replicated branch would be addressed by.

Motivation

The driving use case is isolated application versions: deploy multiple variants of an app to the same Harper instance (or cluster), each with its own isolated view of data, so writes don't collide and tests don't pollute the shared database. Today there is no way to give an app a private writable copy of an existing database short of running a separate instance.

UX

Declared on the application's root-config entry, alongside host and urlPath — which databases an application forks is a deployment decision, not something the application checks in. Declaring it in the application's own config.yaml is refused rather than ignored.

my-app:
  package: my-app
  host: api.example.com
  branchedDatabases:
    - data

branchedDatabases: true forks every database on the instance except system. It is a snapshot at load, not a subscription: a database created afterward is not retroactively branched.

The app then imports from harper normally:

import { databases } from 'harper';
await databases.data.MyTable.put({ id: 'x', ... });

The import is load-bearing. A branch is delivered through the module loader's harper exports; the bare databases/tables globals are shared process-wide by the default vm-current-context loader and cannot be scoped, so an application that reaches for them reads and writes the BASE, silently. Per-application globals are a property of the thread-isolation track, not of branching.

Approach

RocksDB checkpoint + hard-linked blob clone + per-application databases binding.

1. On app load, for each declared database, createCheckpoint() into a staging sibling and rename it into place, so a crash mid-copy leaves debris rather than a half-populated directory RocksDB would refuse to open. Concurrent worker threads settle who takes the checkpoint through a claim word in a buffer the base store shares across threads.
2. Open the checkpoint into a caller-owned table graph — never registered in the process-global databases map. That is what keeps a branch out of describe_all, analytics, worker teardown, and replication by construction rather than by a skip list each future enumerator has to remember. The branch's store carries its own identity, length-prefixed so two (app, database) pairs can never compose the same store name.
3. In getHarperExports(scope), a branched application's databases/tables/defineTable resolve through a live view over the real map. An unbranched application gets the process-wide singletons by identity, so the common path is provably unchanged and only branched applications pay for the indirection.
4. Blobs are hard-link cloned alongside the checkpoint, so the OS inode refcount does the reference counting and each branch keeps its own independent directory and ID allocator. No shared allocator, no high-water mark, no gated deletion, no orphan-GC suspension. See Branched databases: blob store sharing, allocator, and GC safety #644 for the full design.
5. A branch is removed deliberately — undeploying the application — not on process exit. There is no startup sweep, because there are no per-process directories to sweep.

Branches must not participate in cluster replication. Verified: replication reaches tables only through the global registry (getDatabases()[databaseName], dbReplicationWorkers.get(databaseName), the wire subscription's sub.database), and a branch is never in that map. Subscriptions are safe by a different mechanism — addSubscription keys on table.primaryStore.path, and a branch has its own path.

Subtasks

• Expose RocksDB Checkpoint API as createCheckpoint(path) rocksdb-js#577 — Expose createCheckpoint() in the native binding
• Branched databases: app config, scope proxy, lifecycle wiring #643 — App config, root-config declaration, scoped bindings, lifecycle (shipped in Give an application a private, durable fork of a database #2352)
• Branched databases: blob store sharing, allocator, and GC safety #644 — Blob hard-link clone (branch reads of pre-existing blobs are broken until this lands)
• Branched databases: scope the table factory through GraphQL @table, ensureTable, and defineTable #2264 — Scope the table factory through GraphQL @table, ensureTable, and defineTable — PR feat(branches): declare tables into a branch through @table, ensureTable and defineTable #2523 (draft)
• Branch removal on undeploy — removeBranches() exists and is tested but has no production caller, so an undeployed application's fork outlives it — PR feat(branches): remove an application's branched databases on drop_component #2517 (merged)
• Integration tests: isolation in both directions, durability across restart, branchedDatabases: true, and the schema gate
• Tier 2, thread isolation (first piece): isolated: true runs an application in a dedicated worker thread — PR feat(threads): run an isolated application in a dedicated worker thread #2524 (draft); then proxy route composition, ops-API host scoping, log isolation

Known gaps

• Schema is frozen at checkpoint. A branch carries its own copy of the schema as of its creation, and the DDL fence refuses dropTable/addAttributes/removeAttributes/subscribe-auditing/@table/ensureTable/defineTable through a branch, so a branched application cannot evolve its schema at all until Branched databases: scope the table factory through GraphQL @table, ensureTable, and defineTable #2264. Tier-1 applications must ship their schema in the base first.
• A killed claim winner is unrecoverable in-process. A worker terminated mid-createCheckpoint leaves the claim at CREATING with nobody to release it, so later loads of that application burn the full 10-minute deadline. A filesystem-level claim is the durable answer.
• operation() is instance-global by decision — the operations API is administrative, so operation('create_table', …) from a branched application reaches the base.
• Deploy pre-flight validates a candidate against the base (pre-existing): validation loads the component without branch bindings, so a branched application's load-time database work hits the shared database during validation.

Explicit non-goals

• Merge / promote branch writes back to base.
• Per-table (rather than per-database) branching.
• Branch sharing across applications.
• LMDB backend — RocksDB only.

🤖 Updated by Claude on behalf of Kris (2026-08-31) — brought in line with what shipped in #2352: branches are durable with a deterministic, cluster-addressable identity (not ephemeral/per-process), declared on the root-config entry, kept out of the global map rather than namespaced within it, and blob handling is a hard-link clone rather than a shared store with a high-water mark. Original body preserved in a comment.