feat: Generational NodeStore GC for online_delete

[dangell7]

High Level Overview of Change

online_delete rotates by copying every live node of the state map into a fresh NuDB file and then dropping the old one, so the copy is O(total state) and it runs on every rotation. A node that has not been touched in a year is still rewritten every interval.

This replaces the two backend copy with a ring of generations. Writes go to the newest generation, reads walk newest to oldest, and once the ring is longer than its budget the oldest generation is retired: only the nodes it still serves are copied forward, then the whole file is dropped. An evacuated node lands in the newest generation, so it has to age across the entire ring before it can be copied again. A cold node is re-stored once per budget rotations instead of once per rotation, and rotation cost follows churn rather than total state.

online_delete_generations sets the budget. Default 8, clamped to 2..64, documented in cfg/xrpld-example.cfg. The minimum of 2 is one writable generation plus one archive, which is the shape rotation has today.

Context of Change

Rotation is the largest write burst a node does, and it grows with account state. The generational ring is the standard way out: the newest generation absorbs writes, cold data is dropped by unlinking a file instead of being rewritten, and only the survivors of the oldest generation pay a copy.

Measured write volume over 16 rotations against a 16,000 node cold set with 32 churned nodes per rotation, from xrpl.app.SHAMapStoreBench:

ring budget	nodes re-stored	vs copying everything
full copy (today)	256,000
2	128,000	2x fewer
4	64,000	4x fewer
8 (default)	32,000	8x fewer
16	16,000	16x fewer

The reduction factor is the ring budget, and it holds at every cold set size measured. Full matrix, the mechanism, and the method: https://xrpld-online-delete.vercel.app

Trade-off worth naming: a bigger ring keeps more generation files on disk and retires less often, so the knob buys fewer writes with footprint.

API Impact

• Public API: New feature (new methods and/or new fields)
• Public API: Breaking change (in general, breaking changes should only impact the next api_version)
• libxrpl change (any change that may affect libxrpl or dependents of libxrpl)
• Peer protocol change (must be backward compatible or bump the peer protocol version)

DatabaseRotating grows the ring interface (advance, beginRetire, retireOldest, endRetire, generationCount) in place of the two backend swap, and DatabaseRotatingImp's constructor now takes the opened ring instead of a writable plus an archive.

Test Plan

xrpl.app.SHAMapStore: 15 cases, 618 tests, 0 failures. The new cases cover retention across every retirement, refusal to start on a corrupted ring, removal of orphan generation directories at boot, ring survival across a node_db path change, and the write volume claim above.

./xrpld --unittest=xrpl.app.SHAMapStore
./xrpld --unittest=xrpl.app.SHAMapStoreBench   # manual, prints the matrix

The bench drives the real advance / beginRetire / retireOldest path against an in-process backend ring with the production evacuation pattern, so it measures copyForwardCount() rather than a model. It is not a full node measurement: NuDB behavior under real ledger load and a long running node with history are still worth a soak before this is trusted in production.

[xrplf-ai-reviewer]

The generational ring refactor is thorough and well-tested (retention, concurrency, migration, convergence, orphan cleanup are all covered). The core lock/copy-on-write ring logic in DatabaseRotatingImp, the SavedState migration path, and the SHAMapStoreImp retire loop all look internally consistent with the tests provided. One correctness concern worth a second look: forEach() now walks every generation in the ring, which can surface the same logical node twice while a node sits in both a sealed generation and the writable generation after being copied forward but before the sealed generation is actually retired.

[github-actions]

This PR has conflicts, please resolve them in order for the PR to be reviewed.

[Copilot]

🔵 Needs a closer look

It changes core on-disk NodeStore rotation semantics and persistence/migration logic, which is high-impact and warrants final human review despite strong test coverage.

Pull request overview

Implements a generational (ring-based) NodeStore garbage-collection strategy for online_delete to avoid O(total state) full rewrites on each rotation by retiring only the oldest generation and evacuating just its still-live nodes forward.

Changes:

• Replace the two-backend rotate/swap model with an N-generation ring (newest writable, reads probe newest→oldest, retire oldest when over budget).
• Persist the ring durably in the state DB (new DbGenerations table) while keeping legacy {ArchiveDb, WritableDb} in sync for downgrade compatibility.
• Add online_delete_generations config (default 8, clamped 2..64) and extensive unit tests + a manual benchmark suite.

File summaries

File	Description
src/xrpld/app/misc/SHAMapStoreImp.h	Adds generation-budget configuration constants and updates copyNode signature to carry NodeObjectType.
src/xrpld/app/misc/SHAMapStoreImp.cpp	Opens/persists a backend ring, advances generations, retires oldest generations under budget, updates path/orphan handling.
src/test/app/SHAMapStore_test.cpp	Adds ring lifecycle, retention, concurrency, migration, and scaling tests; adds manual benchmark suite.
src/libxrpl/server/State.cpp	Creates/uses DbGenerations; adds atomic (transactional) ring persistence and legacy fallback/staleness detection.
include/xrpl/server/State.h	Extends SavedState with generations and documents legacy compatibility behavior.
src/libxrpl/nodestore/DatabaseRotatingImp.cpp	Implements ring mechanics (advance, begin/end retire, retireOldest) and scoped copy-forward from retiring generation only.
include/xrpl/nodestore/detail/DatabaseRotatingImp.h	Updates rotating DB implementation interface/fields for ring + retiring-generation semantics.
include/xrpl/nodestore/DatabaseRotating.h	Expands public rotating DB API from rotate/swap to ring lifecycle (advance/retire + counters).
include/xrpl/config/Constants.h	Adds online_delete_generations config key constant.
cfg/xrpld-example.cfg	Documents online_delete_generations behavior, defaults, and clamp range.

Review details

• Files reviewed: 10/10 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

All conflicts have been resolved. Assigned reviewers can now start or resume their review.

[codecov]

Codecov Report

❌ Patch coverage is 81.54506% with 43 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/xrpld/app/misc/SHAMapStoreImp.cpp	78.0%	27 Missing ⚠️
src/libxrpl/nodestore/DatabaseRotatingImp.cpp	82.0%	16 Missing ⚠️

📢 Thoughts on this report? Let us know!

[xrplf-ai-reviewer]

No issues found.