Skip to content

Start one stale-while-revalidate refresh per record at a time - #4367

Merged
kriszyp merged 7 commits into
fix/source-fill-lock-tenurefrom
fix/stale-refresh-once-per-key
Nov 14, 2026
Merged

kriszyp merged 7 commits into
fix/source-fill-lock-tenurefrom
fix/stale-refresh-once-per-key

Conversation

@ldt1996

@ldt1996 ldt1996 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Lavinia shows a one-runner turnstile to the refill pipe

⊙ Problem

When allowStaleWhileRevalidate serves an expired record, every read also queued its own background refresh behind the running one. If the source kept failing, each queued refresh called it again and logged a warning: one source call and one warning per read. 300 stale reads of a record whose source fails after 150 ms made 300 calls and 300 warnings over 45 seconds.

💡 Solution

  • A stale read starts a refresh only if no other request holds that record's fetch. Otherwise it returns the stale record.
  • The read after a failed refresh starts a new one.
  • allowStaleWhileRevalidate now runs before the refresh starts.
  • Reads that cannot be served stale still wait for the fetch.
  • Stacked on Release a source fetch's record lock only from the fetch that took it #4366, whose 10-second hold limit frees a record whose refresh never settles; this PR's base is that branch until it merges.

🤔 Skip, or queue one retry?

A stale read that finds the record busy skips the refresh, so after a failed refresh the record stays stale until the next read. Queueing one retry per record would refresh sooner, at the cost of a waiting request per record. Switching later touches only the stale path.

⚖️ Alternatives

The planning review found a better approach than the one first proposed, a hasLock pre-check: take the lock atomically with a callback-free tryLock, and refresh only on success. This PR adopts it.

Framing-Verdict: better-alternative-exists

  • Check hasLock, then lock inside the fetch — two threads can both see the record free and both refresh.
  • Back off after a failed refresh — a freshness policy; a separate design.

🔧 Changes

Product and architecture tour

Product and architecture tour

One refresh per expired record

What does a read of an expired record do when stale serving is allowed?

Before After
Every stale read returned the stale record and queued its own refresh behind the running one; after a failure, each queued refresh called the source again. A stale read refreshes only if it can take the record lock at once; otherwise it returns the stale record and does nothing more.

A stale read, step by step

  1. Ask the hook — allowStaleWhileRevalidate decides whether this read may be served stale.
  2. Try the lock — a callback-free tryLock, so a busy record registers no waiter.
  3. Busy: return stale — another request's fetch owns the record; nothing is queued.
  4. Free: refresh in the background — the read hands the held lock to getFromSource and returns the stale record.
  5. Release — the fetch releases the lock at commit, or when its hold ends if the source never settles.

A read that may not be served stale still waits for the fetch, as before.
Record locks are per database on RocksDB, so a fetch of the same id in another table of the database also makes a stale read skip; that keying is tracked separately.

✅ Verification

  • Fails on main, in unitTests/resources/staleRefreshOncePerKey.test.js: reads queue a refresh each; the query path makes 2 calls, expected 1; a record another thread holds is refreshed anyway.
  • 8 of 8 pass on RocksDB and LMDB, including a worker thread holding the record lock and a refresh that never settles. Without the setup-throw release, its test fails. The full unit suites fail only tests that also fail on main on the same machine; one record-lock test that failed in this branch's RocksDB run is flaky on main too (2 of 10 runs alone fail there, 0 of 10 here).
  • 4 threads, 300 stale reads, refresh failing after 150 ms: main made 300 source calls and logged 300 warnings over 45.8 s; this branch made 1 call and logged 1 warning in 0.2 s. All reads were answered from stale within 4 ms on both.
  • Performance: fresh hits do no new work. A stale read adds one tryLock, and a busy record skips the refresh setup.
  • PR QA: skipped @ 8414d3c — Dispatch is not available in this session.
Test cases
  • Concurrent stale reads start one refresh; after it fails, no read had queued its own, and the next read refreshes.
  • A successful refresh replaces the record.
  • A read that may not be served stale still waits for the refresh.
  • Concurrent queries over a stale row start one refresh.
  • A tryLock that throws is logged, and the read serves stale.
  • A context that throws once the lock is held releases it; the next read refreshes.
  • A refresh whose source never settles keeps the record busy only until its hold limit; the next read refreshes again.
  • A worker thread holding the record lock: the read serves stale and queues nothing; after the release, a read refreshes.

🤖 Generated by Claude Opus 5.5 (Claude Code); posted via @ldt1996.

Release target: v5.4 — main is the 5.4 line; fixes stale-while-revalidate behavior that shipped in 5.4.0

Related PRs: #2155 overlaps (its drain may assume getFromSource takes the fill lock), #2918 overlaps (a lock handed in by a stale read skips the acquisition it orders), 5 others independent
Complexity: complicated

Deferred findings

  • 78defa8349fe2006 minor correctness — resources/Table.ts — Database-wide bare-id fill lock makes a stale read skip its refresh when another table fills the same id — NOT FILED
  • 6aef72546c35fcc8 nit correctness — resources/Table.ts — Background refresh .then mutates the leading stale read's context after it returned — NOT FILED
  • 210a587193a7fbb1 nit maintainability — resources/Table.ts — New comments in the lock-tenure prerequisite (Table.ts:461, :14338, sourceFillLockTenure.test.js) — NOT FILED

Review-Coverage: authored=claude; ran=codex,gemini,gemini-carbon; late=gemini-carbon:adjudicated; adjudicated=domain; blocked=cursor-composer(not-installed); declined=cursor-grok,muse; rounds=8; full=3 @ 8414d3c

Review-Attention: deep ~35m (critical: Table.ts; decisions: stale-busy-skip, waiters-never-unlock, fixed-10s-tenure, do-less-alternative, release-target-unclear; raised: degraded review) @ 8414d3c

Lavinia and others added 7 commits November 13, 2026 17:16
ensureLoadedFromSource started getFromSource for every read of an
expired record before asking allowStaleWhileRevalidate. While one
refresh ran, every further stale read queued another background fill on
the record lock; when the refresh failed, each queued fill called the
source again in turn and logged its own warning.

The stale policy is now decided first. A read that serves stale
refreshes only if it acquires the record lock: getFromSource tries the
lock without a callback and, when another fill (on any thread) holds
it, returns without parking, arming a timer or touching the request's
context. Using the acquisition itself as the gate leaves no window
between a check and the lock. Reads that must wait for a fresh value
are unchanged, and the read after a failed refresh starts a new one.

Refs #1398

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stale read now tries the record lock synchronously, without a
callback, before getFromSource runs: when another fill holds the key it
returns at once, minting no timestamp, encoding no key and allocating no
promise. getFromSource takes over the acquired lock (lockHeld) instead
of trying it again, and the FILL_IN_FLIGHT sentinel is gone. A tryLock
that throws (a closed store) skips the refresh rather than failing the
read. The docs now promise only what the lock gives: no other request
is holding the record's fetch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, and report an untryable lock

A stale read takes the record lock before getFromSource runs, so a
throw while reading the context's timestamp or encoding the key would
have left that lock with no owner to release it; getFromSource now
releases a handed-over lock when that setup throws. (A throw from the
issuance fallback means the store is closing, and closing releases the
handle's locks.) A tryLock that throws is logged as a warning, as it
was when the attempt ran inside getFromSource, and the read serves
stale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ngines

A context whose timestamp read throws once the stale read holds the
lock now reaches the release in getFromSource; without that release
the next stale read never refreshes. The other-thread test takes the
record lock from a worker thread instead of a second handle on this
one, so it runs on LMDB too. Teardown waits for every refresh to
commit and release its lock. A moved comment that restated the code
is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5.4.0 shipped before this change, so it lands in a 5.4 patch release,
and patch releases are described in GitHub releases, not in fragments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er inert under mocha

Stacked on the fill hold limit, a stale refresh whose source never
settles now holds the record only until the limit; a test leaves one
refresh unsettled and checks the next stale read refreshes after it.
The worker helper does nothing unless the test passes workerData, as
mocha's resources glob also loads it on the main thread, where it threw
and aborted the run. The setup guard now also covers issuing the source
version, whose fallback can throw on a closing store. The comment above
the stale admission is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kriszyp kriszyp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Reviewed with Muse Wonder 1.0 · adjudicated by Codex

}
});

it('starts no refresh while another thread holds the record lock', async function () {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Possible LMDB-only failure in the foreign-lock test remains disputed

@kriszyp
kriszyp merged commit e445fc6 into fix/source-fill-lock-tenure Nov 14, 2026
96 of 101 checks passed
@kriszyp
kriszyp deleted the fix/stale-refresh-once-per-key branch November 14, 2026 14:56
kriszyp pushed a commit that referenced this pull request Nov 14, 2026
…#4366)

* Add a test seam for the source-fill lock timeout

setSourceFillLockTimeoutForTesting() lowers how long a request waits on a
source fill's record lock before its timeout fires, so a test can reach
that path without waiting ten seconds. Behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Release a source fill's record lock only from the fill that took it

A request waiting on a sourcedFrom fill's record lock unlocked it after
LOCK_TIMEOUT, and neither engine's unlock checks ownership. When the
source took longer than that (a slow origin), a waiter released the
lock its live fill still held, a second fill started, and the first
fill's own release later freed the second fill's lock, waking every
waiter again.

The fill that acquires the lock now releases it exactly once, through
an idempotent releaseFill: when its commit settles, or when its tenure
reaches LOCK_TIMEOUT, so a source call or commit that never settles
still frees the key. A waiter's timer only rechecks hasLock() and keeps
its one registration instead of unlocking or re-queueing; a closed
store makes hasLock() throw, so a waiter whose own handle closed still
settles. A holder that goes away releases its locks natively: closing a
RocksDB handle (lockReleaseByOwner) or an LMDB environment wrapper
(closeEnv) frees them and wakes their waiters.

Refs #1398

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Cite the close test in the record-lock note and guard the tenure test's teardown

The note's close half is still enforced by cacheFillClose.test.js. The
test file's header comment said a waiter only wakes on its timeout, but
an unlock wakes it too, so it is gone; afterEach no longer assumes
beforeEach ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Badge the change for 5.4.1 and drop its 5.4 release-note fragment

5.4.0 shipped before this change, so it lands in a 5.4 patch release,
and patch releases are described in GitHub releases, not in fragments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Let the tenure tests' teardown wait for every fill they started

A read left unsettled by a test, such as the stuck first fill, resumed
after the hooks returned and committed into the next test's database.
Teardown now settles the reads, waits until their records are
unlocked, and waits for pending commits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Start one stale-while-revalidate refresh per record at a time (#4367)

* Start one stale-while-revalidate refresh per record, not one per read

ensureLoadedFromSource started getFromSource for every read of an
expired record before asking allowStaleWhileRevalidate. While one
refresh ran, every further stale read queued another background fill on
the record lock; when the refresh failed, each queued fill called the
source again in turn and logged its own warning.

The stale policy is now decided first. A read that serves stale
refreshes only if it acquires the record lock: getFromSource tries the
lock without a callback and, when another fill (on any thread) holds
it, returns without parking, arming a timer or touching the request's
context. Using the acquisition itself as the gate leaves no window
between a check and the lock. Reads that must wait for a fresh value
are unchanged, and the read after a failed refresh starts a new one.

Refs #1398

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Admit a stale refresh before starting it, so a skipped read does no work

A stale read now tries the record lock synchronously, without a
callback, before getFromSource runs: when another fill holds the key it
returns at once, minting no timestamp, encoding no key and allocating no
promise. getFromSource takes over the acquired lock (lockHeld) instead
of trying it again, and the FILL_IN_FLIGHT sentinel is gone. A tryLock
that throws (a closed store) skips the refresh rather than failing the
read. The docs now promise only what the lock gives: no other request
is holding the record's fetch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Release a lock a stale read handed over if the refresh fails to start, and report an untryable lock

A stale read takes the record lock before getFromSource runs, so a
throw while reading the context's timestamp or encoding the key would
have left that lock with no owner to release it; getFromSource now
releases a handed-over lock when that setup throws. (A throw from the
issuance fallback means the store is closing, and closing releases the
handle's locks.) A tryLock that throws is logged as a warning, as it
was when the attempt ran inside getFromSource, and the read serves
stale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test the stale refresh's lock release and other-thread skip on both engines

A context whose timestamp read throws once the stale read holds the
lock now reaches the release in getFromSource; without that release
the next stale read never refreshes. The other-thread test takes the
record lock from a worker thread instead of a second handle on this
one, so it runs on LMDB too. Teardown waits for every refresh to
commit and release its lock. A moved comment that restated the code
is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Badge the change for 5.4.1 and drop its 5.4 release-note fragment

5.4.0 shipped before this change, so it lands in a 5.4 patch release,
and patch releases are described in GitHub releases, not in fragments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Free a hung stale refresh at its hold limit, and keep the worker helper inert under mocha

Stacked on the fill hold limit, a stale refresh whose source never
settles now holds the record only until the limit; a test leaves one
refresh unsettled and checks the next stale read refreshes after it.
The worker helper does nothing unless the test passes workerData, as
mocha's resources glob also loads it on the main thread, where it threw
and aborted the run. The setup guard now also covers issuing the source
version, whose fallback can throw on a closing store. The comment above
the stale admission is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Drop a test comment that narrated the teardown wait

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Lavinia <lavinia@Lavinias-MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Lavinia <lavinia@Lavinias-MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants