Skip to content

serve: prune cached hashes to bound cache growth#417

Merged
tinder-maxwellelliott merged 1 commit into
masterfrom
claude/cache-pruning-serve-132221
Jul 9, 2026
Merged

serve: prune cached hashes to bound cache growth#417
tinder-maxwellelliott merged 1 commit into
masterfrom
claude/cache-pruning-serve-132221

Conversation

@tinder-maxwellelliott

Copy link
Copy Markdown
Collaborator

What

The serve query service caches one hash entry per queried commit SHA under --cacheDir and never removed any, so a long-running server's cache grew without bound. This adds optional, off-by-default pruning that bounds the on-disk footprint.

New serve flags (all optional; unset = unlimited):

Flag Meaning
--cacheMaxAge Evict entries not read/written within a window, e.g. 7d, 36h, 90m.
--cacheMaxEntries Cap the number of cached commit-SHA entries (LRU).
--cacheMaxSize Cap total on-disk size, e.g. 10GB, 500MB, or a bare byte count.
--cachePruneInterval How often the sweeper runs. Default 1h.

With no --cacheMax* flag set, the cache is never pruned (prior behavior).

How

  • PrunableHashCacheStorage capability interface (extends the byte-oriented HashCacheStorage). LocalDiskHashCacheStorage implements it: LRU by last-use — get() bumps the file's mtime on a cache hit, so a base revision under active query is not expired out from under live traffic — and prune() evicts oldest-first by any combination of age / count / size. get() is also hardened against a file pruned out from under a concurrent read.
  • CachePruner — a single daemon ScheduledExecutor that sweeps once at startup, then every --cachePruneInterval (fixed delay, so a slow sweep can't pile up). No-op when no limit is set; never throws, so a transient FS error can't kill the sweeper thread.
  • DurationConverter / ByteSizeConverter — small reusable picocli converters for the human-friendly flag values.
  • Wired into ServeCommand's startup/shutdown lifecycle. A backend that can't self-prune (a future S3 store, which would use a bucket lifecycle policy) is detected and the --cacheMax* flags are reported-and-ignored rather than silently pretended-to.

Notes for reviewers

  • Pruning flags are deliberately not part of the config fingerprint — retention doesn't change hash content, and folding them in would needlessly partition the cache.
  • The base HashCacheStorage interface stays byte-oriented so a remote backend still drops in unchanged (RFC RFC: Bazel Query Service #29 S3 direction).

Testing

  • Unit tests: prune-by-age/count/size on a real filesystem, touch-on-hit LRU, non-.json files left alone, both converters, and CLI wiring/parsing (incl. the 1h default parsed through the converter).
  • End-to-end: a test that drives the real scheduler against real on-disk storage and asserts files are deleted down to the limit.
  • Real-bazel E2ETest#testServeEndToEnd (exercises the modified cache-hit path) passes. ktfmt/buildifier clean.
  • Docs updated in tools/readme_template.md + regenerated README.md.

The serve query service wrote one cache entry per queried commit SHA to
--cacheDir and never removed any, so a long-running server's cache grew
without bound.

Add optional, off-by-default pruning:

- PrunableHashCacheStorage capability interface + LocalDiskHashCacheStorage
  impl: LRU by last-use (get() touches mtime on a cache hit), evicting
  oldest-first by any mix of max age / max entry count / max total size.
  get() is hardened against a file pruned out from under a concurrent read.
- CachePruner: a daemon scheduler that sweeps once at startup then every
  --cachePruneInterval (default 1h); a no-op when no limit is set; never
  throws, so a transient FS error cannot kill the sweeper thread.
- serve flags --cacheMaxAge (e.g. 7d), --cacheMaxEntries, --cacheMaxSize
  (e.g. 10GB), --cachePruneInterval, plus two reusable picocli converters
  (Duration, byte size). Limits are deliberately NOT folded into the config
  fingerprint (retention does not affect hash content). A limit set against a
  non-prunable backend (a future S3 store) is reported and ignored rather than
  silently pretended-to.

With no --cacheMax* flag set the cache is never pruned (prior behavior).

Tests: prune logic on a real filesystem, touch-on-hit LRU, both converters,
CLI wiring, and an end-to-end test that drives the real scheduler against real
on-disk storage and asserts files are deleted down to the limit. Docs updated
in readme_template.md and the regenerated README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tinder-maxwellelliott tinder-maxwellelliott marked this pull request as ready for review July 9, 2026 16:19
@tinder-maxwellelliott tinder-maxwellelliott merged commit 570a90a into master Jul 9, 2026
15 checks passed
@tinder-maxwellelliott tinder-maxwellelliott deleted the claude/cache-pruning-serve-132221 branch July 9, 2026 16:19
tinder-maxwellelliott added a commit that referenced this pull request Jul 9, 2026
Resolves conflicts from #417 (serve cache pruning) landing on master:
- HashCacheStorage.kt: LocalDiskHashCacheStorage now implements BOTH
  MeasurableHashCacheStorage (stats(), this PR) and PrunableHashCacheStorage
  (prune(), #417), keeping both method bodies and the shared touch-on-get LRU.
- ServeCommand.buildAndStartServer wires the MetricsService alongside #417's
  CachePruner startup; imports for both features merged.
- cli/BUILD and LocalDiskHashCacheStorageTest keep both features' test
  targets and cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant