diff --git a/.changeset/preserve-reentrant-ordering-repair.md b/.changeset/preserve-reentrant-ordering-repair.md new file mode 100644 index 0000000000..4b259def0f --- /dev/null +++ b/.changeset/preserve-reentrant-ordering-repair.md @@ -0,0 +1,5 @@ +--- +'@tanstack/db': patch +--- + +Preserve ordering invalidations raised while an ordered repair retires its previous acquisitions. Revalidate the affected source prefix before publishing the repaired window. diff --git a/docs/contributing/oracle-coverage.md b/docs/contributing/oracle-coverage.md index a6616dd4df..14e8992221 100644 --- a/docs/contributing/oracle-coverage.md +++ b/docs/contributing/oracle-coverage.md @@ -568,7 +568,7 @@ that test identifiers must copy production's private data structures. | Drafts and native values | Complete | Native differential behavior, draft change tracking, detachment and its class-instance exception, hostile keys, aliases, cycles, and Map/Set live iteration are literate. | | Query DB and observer | Complete | Query-scope row ownership, subset identity and cancellation, failure and recovery, and the per-listener eligibility ledger are literate. | | Query DB SSR dehydration | Fixed QueryClient/Seroval boundary | Successful on-demand entries with a compound predicate, a request signal, and an ordered cursor with a custom comparator retain their row and user metadata through Seroval's reconstructed stream payload and Query Core hydration; request options are absent from the stream, while query functions retain the original values. The TanStack Start router and browser Collection resume path still need a host witness. | -| Ordered acquisition | Complete | Exact demand identity, applied settlement, independent pagination recomputation, request work, lifecycle products, replay authority, source-generation readiness, and transaction-refinement abort boundaries are literate. An initial full-source ordered load shares the synchronous readiness cut ([historical review](oracle-reviews/initial-full-source-sync-readiness.md)). The ordered-lifecycle oracle covers an inexpressible-boundary fallback before first publication when every acquisition returns literal `true`. It also checks a later null-boundary fallback after the first joined window publishes, across literal, Promise, and held-receipt responses, including a held receipt after child demand settles ([review](oracle-reviews/issue-2152-initial-ordered-fallback.md)). The ordered loader state oracle also has a generated request-authority lane (`ordered-loader.request-authority`, fixed and random campaigns). It drives the real loader over a fake subscription through settlements in any order, failures, deferred invalidation and report reactions, ordinary passes and window moves, and began with four laws from a TLA+ model (#2191): a successful repair keeps its acquisitions until a covering read succeeds; quiet loading holds nothing older than its last authoritative read; a superseded request's failure is recovered by a full-source read without an explicit retry; and once quiet, an invalidation and a pass issue a read unless a full-source read covers the source. Scenarios also throw from the first release callback, answer a repair synchronously from installed rows, and let a page's own delivery invalidate ordering; named replays keep the traces they found, each of which failed on main (a throwing release that unwound a resumed repair, a synchronous repair waiting behind a superseded request, and a self-invalidating page whose tie was never retired). On main it found a superseded tie that failed after its repair finished and a newer invalidation's reaction was deferred, which left refills blocked; a named replay keeps that trace. A stale prefix that retires the repair's prefix fails the held law, and a stale request held after the repair finished fails the leak law. The lane also crosses unexpressible boundaries, joined work that defers loading, truncate replay that succeeds or stays pending, optimistic departures, source deletes, the bounded backoff retry, window moves that shrink or widen, plans that read only the full source, and reports inside the loader's own first page. A replay owns the requests issued before its truncate, and when the subscription still holds a full-source lease the settled replay covers the source. These dimensions found three more defects, each kept as a named replay: a repair prefix that settled after a replay started its replacement took the replacement's chain tag and was never retired; a replay counted a full-source read as complete after an explicit retry had released it; and a failure from before a truncate dropped the next repair's staged step. Each of the seven defect replays fails on the loader at the branch's merge base with main. A later law, one request at a time, found that a replaced repair chain's prefix still counted as current and issued a tie beside the replacement's prefix; a chain is now superseded by any newer authority start. The failed-lease law also counts a covering replay, which a named replay reaches. Repeated random campaigns then found a repair request that settled after full-source coverage and stayed held; full coverage now retires it. The held law counts a later prefix that a successful replay reacquired while it was in flight. Four histories from TCP's known error classes followed: a request the provider never answers (a lost segment), a release callback that truncates synchronously (a reentrant reset), a window offset other than zero, and an at-most-once release law with no release while a replay owns the demand unless an explicit retry replaces it or a held full-source lease covers it. They found that the backoff retry's release pass kept releasing failed demands after a release started a truncate; every release pass now stops on a reentrant reset. The oracle now observes epoch moves as calls to `invalidateSourceOrdering()`, so it tells a replacement chain from a chain step exactly. The loader's work slot and debt now move only through a transition table that throws `OrderedLoaderInvariantError` on an illegal move or an incoherent work and debt pair; the oracle records any such error as a broken law, so its generated histories fuzz the table. A covering-lease law (a full-source lease whose read succeeded or whose demand a replay reacquired is never released) found that a full-source transport failing after a covering replay turned complete coverage into an own failure; the table then found that a repair request failing after a replay finished its chain did the same. Mutants that drop each new guard fail. A release callback that runs an ordinary pass synchronously, as a graph run does when the release delivers deletes outside a transaction, found that the pass issued a page while another request was settling, and that request then issued its tie; every release pass now blocks refinement until it unwinds. A dispose command and a release callback that disposes the loader check that a disposed loader issues and releases nothing; mutants that drop the loader's own `active` checks in the release pass and in failure handling survive because dispose resets the cursor, so those checks were removed. TLA+ run 3 (`LoaderTable.tla`) models the table with truncate and replay; its safe rules pass with 9,368,122 distinct states, and each of the four guards from this round, removed alone, produces a counterexample. Reverting any one fix fails its replay and a campaign. The generated histories also stack a second truncate on a pending replay and truncate after a failed replay, each with a reach witness. A lease missing from its map during a release pass is now an invariant violation; no history reaches it. A second high review found that a covering replay did not retire a superseded failed lease from before its truncate, because the failed-lease law exempted every replay-owned request. The law now judges a superseded failure that was live (it failed in the incarnation it was issued in) once a covering replay settles after it, and the replay retires those leases. The subscriber now schedules the replay's graph run in a finally. The replay-refinement oracle is the receiving witness: its generated ordered section (`replay.ordered-window-faults`) drives a real on-demand Collection and an ordered live query through rank moves, held and failed loads, truncates, and unloads that throw without a truncate, while a replay starts, and while it settles. Its settled-window law found that a release throwing after a successful ordered load left the window on old rows, because the builder counted the rejected participant as a failed load (also on main); the loader now marks release errors and the builder publishes. It also fails without the subscriber's finally. The section now also crosses tied ranks (its law compares the window's rank sequence and each shown row's current rank), a peer live query on the same source, offset 1, descending order, and explicit window moves; a move that rejects after a failed source recovery leaves the window, and one rejected only by a release failure still moves. A reported-faults law accepts injected faults as rejected loads, thrown releases, rejected moves, or errors the subscription rethrows from a microtask, and rejects any other escaped error. These found that a window move whose load succeeded was recorded as failed when a release threw, so the window stayed private until another move. A throwing release is an adapter bug, so the loader now reports it uncaught from a microtask and changes nothing else; the reported-faults law rejects a move that a release failure rejects, and the request-authority oracle requires each throwing release to be reported uncaught exactly once (a mutant that swallows the error fails). It also crosses a joined marker source: a LEFT join that keeps marked or unmarked rows, and an inner join. The marker source is eager or on-demand, with marker loads that are held, fail, and truncate. This found that an on-demand joined source whose loads finished without delivering a row left the window on old rows, because nothing ran the graph that the pending joined work held; the subscriber now runs it when a joined source becomes ready. A truncate never aborts a window move: the move waits for any replay that holds publication and then applies (ARCHITECTURE.md), so a move that rejects with `AbortError` during a history is an escaped error. (An earlier version of this entry claimed a named replay reached an aborted-move branch; no history reached it, and the branch, which quietly accepted such a rejection, is gone.) Recovery truncates the marker source too, because a failed marker replay keeps the last complete publication until that source replays. A hostile-mutant pass over the loader's guards found guards that no ordered oracle kills. A differential instrument then ran 3,000 generated request-authority histories through the loader and each mutant and compared every request's shape, outcome, and release. Three guards the code makes redundant are deleted: the backoff timer guard, the explicit-move check in the retry callback, and the `requiresFullSource` check on a report. An ignored-report law now kills two of the mutants that changed public requests: a report issues no authoritative request before any request settled (a request in flight still owes an exact result for its own options) or while the window is inside the loader's own first prefix. Named replays reach both, and the random campaigns judge them in quiet histories. A random ordered-work campaign later found that the first-prefix check was unsound: it took the next installed row as the next source row, but a signalling refetch moved the page's end before the tie the loader had read, so that tie's row stood past the page while a row tied with the new end was never sent, and a later window move skipped it. The loader now accepts the row only when its page read it or when it read the tie of the page's current end. When the end moved, it repairs (see the later maintainer decision below); query-db's first-page witness requires bounded reads only, no full-source read, and an unchanged count of active Queries. The optimistic-window cases that allow but do not require a refill still bound any refill that occurs: an ordered read within the refill bound or a predicate read, never the full source (a mutant that made the growth read unbounded fails them). The ordered-work model now sorts `undefined` ranks with `null` ones, as the comparator does, and its histories generate both; nullish ranks tie in its window and in-flight comparisons. An `undefined` end also matches a loader that attempted no tie; a nullish end is never an expressible cursor, so the loader has started or will start a full-source read, which covers the source either way. A guard against that match survived the generated campaigns as an equivalent mutant and was removed. The ordered-work oracle keeps the trace as a named replay that fails on main, and the request-authority oracle has an unproven-report law (a report over a first page that no row it read proves starts a repair or reads the end's tie), whose replay and fixed campaign fail on main; the broad campaigns reach that law in about one history in 10,000, so an unproven-report campaign (`ordered-loader.unproven-report`, fixed and random) settles the first page, settles the read after it, and reports, over random case parameters and a random suffix, and declares its reach (about a third of 300 histories); `main`'s loader fails both runs. Its fake subscription now reads installed rows as the subscription does, applying a tie's equality, a cursor, and a limit; when its reads ignored the limit, the tie was always the last installed row's, so the first-prefix ignore branch was unreachable. The cost law's per-chain bound, three requests, was an artifact of that fake: it is now a prefix plus a tie and a page per installed row, and it also judges an unindexed window that still needs rows, which grows its prefix instead of reading the full source. The report campaign (`ordered-loader.unproven-report`, 1,000 runs; at 600 a random seed could miss the minimum) now starts each history with one of eight prefixes and declares reach for each report premise: a proven first page, a page that read past a window that shrank, a held full-source read, a head insert that moved the page's end, an owed repair, authoritative work in flight, a release of the older tie that truncates or disposes the loader, and a report to a disposed loader. The unproven-report law requires a bounded repair or the end's tie when every installed rank is expressible; a full-source read no longer satisfies it (a mutant that forced full debt after a report passed before). The cost law also rejects a repeated read: within one invalidation, no prefix, tie, or page repeats more often than the ordering epoch moved. Two laws are stronger: an ignored report, over a proven first page or a held full-source read, issues no request at all, where it used to forbid only authoritative ones; and a report while a repair is owed reads no tie or page before the repair. Deleting the moved-end early return in `revalidateLoadedCoverage`, or its past-window term, fails the first law, and deleting the owed-debt term in `loadMovedEndTie` fails the second; a deletion sweep had found all three surviving every oracle. With that reach, the report path's own `!active` checks were deleted: `dispose()` advances the generation that the stale-run check after the release reads, and a disposed loader issues nothing, which the dispose law judges. A head insert can carry a `-Infinity` rank, and two more laws judge every request: the loader never reads the tie of a rank no cursor can express, and the first finite request after a truncate has no cursor. The fake's `dataNeeded` now counts the window rows not at or before the boundary it is given, where it returned a constant. With those, the stale-run check after the older tie's release fails the reset law when a release truncates the loader and the window then grows (a prefix declares that reach), and removing `loadBoundary`'s own unexpressible-value check fails the unexpressible-tie law. The covered-source early return and the staged-continuation term were deleted: coverage that a full-source read completed holds the work slot, whose check then decides the report, and a staged finite continuation cannot coexist with a proven end, while a staged repair step owes a repair. The ignored-report law's full-source branch no longer needs a quiet history, since such coverage owes nothing, and a prefix reaches a paged loader whose repair fell back to a full-source read; removing the work-slot check then fails it. The unexpressible-end term in `loadMovedEndTie` stays: the unindexed-refetch random campaign found that without it an unindexed loader reads the full source through the boundary fallback where its bounded prefix repair would do, and a named replay of that trace fails without the term. Maintainer decision, later: a report whose page end moved now repairs instead of reading only the new end's tie, to cut shipped bytes; `loadMovedEndTie` and its guards are gone, so their mutants above no longer exist. The laws stay and judge the repair path: the ignored-report, owed-repair, unexpressible-tie, and reset laws all still run, and the unproven-report law accepts a repair. The report campaign drops its two prefixes whose release of the older tie truncated or disposed the loader, since a report no longer releases anything, and the query-db replay of an insert into a top-N list now expects the bounded repair's prefix and the new end's tie. Still open there: the `Object.is` term of the moved-end check. It skips work when the loader already read the end's tie, and it decides differently from its fallback only while other work is in flight, which the ignored-report law's quiet premise excludes. The ordered-work oracle owns that branch instead: peers over a refetching provider stay in its grammar for source filters (#2155 excludes only joined filters), and a proven-report law requires that a report which changes nothing the first query's own page and end tie prove, with the model's next row installed and no installed row deleted or reordered, starts no request for it and leaves it owing no `refetch: true` repair. A campaign fixed to that premise must reach it, and a loader that repairs every report fails both its fixed and random runs. A staged end tie that pending joined work defers is read by the pass after that work finishes; a request-authority named replay reaches it, and a loader that drops the staged tie fails it. The unproven-report law judges that later pass, and a truncate, failure, move, or disposal in between voids what the report owed. A history that starts during the initial preload found that a window move deferred behind a marker replay, which the source truncate then discarded, rejected with that replay's `AbortError`, so the older move's window won (also on main). A deferred move now waits for every replay that holds publication and applies unless one failed or cleanup ended the sync run; a named replay fails without the fix. The ordered-work refetch law also required no full-source read after a signalling refetch for an unindexed plan, which the architecture then let refill a short window from the full source; random seeds found it on main. Maintainer decision on this branch: an underfilled unindexed prefix never reads the full source, because that read is unbounded. It grows instead: each growth asks for twice the last prefix, and it stops once the last growth added no rows. A deletion sweep showed that the older condition's other term, a prefix that reads short, follows from that one. The refetch law now forbids a full-source read after a signalling refetch with no unindexed exception. The named replay of a short unindexed window and the fixed unindexed-refetch campaign (`ordered-work.eager-unindexed-refetch`, fixed and random, with reach on windows the prefix fills and windows it leaves short) failed on the full-source fallback and pass with growth; the full-source mutant still fails them. Growth made a new optimistic history possible: an unindexed one-row window behind a joined or anti-joined filter that installs only the prefix it grew to, whose departure leaves the replacement row missing. Sixteen fixed optimistic-window cases (update and delete, both consumers, acknowledge and rollback) cover it, with a refill bound of twice the installed prefix plus the overlaid row. A loader without the stop rule never reaches a fixed point; a loader that stops whenever the prefix reads short leaves those windows empty after settlement, because the pending departure hides a row from the count. The existing unindexed optimistic cases now allow, but do not require, a request behind a joined filter when the replacement row is installed: growth proves rows only up to its prefix, where the full-source fallback had proven every row (a maintainer-approved expectation change). The proven-report random campaign now declares its reach too. One mutant still changes requests without a law: dropping the optimistic-refill flag turns a refill into a refetching prefix. Six more are equivalent in the fake but stay, because the fake cannot reach their premise or they encode a documented rule: the own-first-page check behind a held sync transaction (the fake never holds one), a truncate moving complete coverage back to held, clearing the pending request when a full-source read starts (the fake never answers a full-source read synchronously), staging a joined `more` continuation, the staged-continuation guard before an unindexed prefix, and the drain re-entry guard (a long synchronous chain would otherwise recurse). An include dimension adds a `toArray` child that reads the marker source for each shown row, and an include-children law requires each shown row's markers to equal the model wherever the window law holds; a mutant that never loads newly demanded child keys fails it in an include-only scenario. A newest-move law requires a superseded window move to resolve only after the newest move settles (a superseded move that fails still rejects with its own error), and the model lets the newest move that resolved set the window; it found that a superseded move resolved as soon as its own load settled, so awaiting it showed neither window. The terminal-error law also allows `error` after a load fails while the query is still loading, the documented initial-load failure; a lazy join source is exempt. Limits: includes are one level deep. It also found that a release failure in lazy joined demand during a window move rejected the move, because the subscription reported it as a load error. A window law now lets the newest move win; it found that two moves deferred by a pending source recovery could resume in the opposite order, so the older request replaced the newer window. `setWindow()` now ignores a deferred call that a newer call superseded. A terminal-error law allows a final `error` only after a lazy join demand load, on either side, failed outside a truncate's recovery; named replays reach that branch. A load belongs to a recovery when it asks again for a demand its source held before the truncate (the same request data), or when the graph starts it while the truncate commits. The first model read a flag set only during the commit, but subscriptions start replay loads in a later microtask, so it counted every replay load as new demand and allowed `error` after any failed replay load; a random campaign found the reverse case (#2224): a marker truncate privately moved the window, the include's new demand started outside any replay because releasing its old demand ended the subscription's replay, and its failure ended the query. The demand controller now starts new coverage before it releases retired coverage, a query subscription opens a replay at every truncate even without demand (a replay that starts no recovery work and reports no loading), and a demand failure that fails its subscription's replay keeps the last publication instead of ending the query once the query has first become ready. The section also generates histories that start during the initial preload, with an initial-readiness law (the preload has settled once loading is quiet), marker loads whose aborted requests reject with `AbortError` as the source contract asks, and settles of the newest held load. These found two older defects on main: an aborted lazy demand request that rejected after its truncate replay succeeded ended the query, and so did a superseded segment that the controller released; the controller now treats an aborted request as obsolete demand, which its replay or newer coverage decides. Named replays keep each trace; reverting each production change fails one of them, and a mutant whose replay rebuilds its request data fails the campaigns' replay-load reach witness. The ordered-lifecycle oracle owns the truncate-work witness: a zero-limit window issues no request and reports no loading at a truncate, while a one-row window reacquires. A retention law then judges law 9 at generated checkpoints: a truncate raises a publication barrier, every load its source starts while the barrier is up joins it, a participant's failure fails the replay while its logical demand is held, and while any barrier is up the query shows exactly the rows it showed when the first one rose. It runs after every command and once loading is quiet. A failed-replay campaign (`replay.ordered-failed-replay`, fixed and random) starts each history by holding loads, truncating a source, optionally starting demand, and failing a held load, with a branch that fails an include demand while its marker replay still loads; it declares four reach states with `createOracleReach` (a judged failed replay, a judged held replay, a demand started after the commit that fails while its replay loads, and the include form of that demand). A replay participant now also counts as recovery for the terminal-error law, so a demand that starts after the commit while its replay is open is judged in generated histories, not only by a named replay. A mutant whose failed replay releases the publication gate fails both failed-replay campaigns and the broad campaigns; a mutant that ends the query when such a demand fails fails both failed-replay campaigns; the branch's production files reverted to `main` fail both. A preload-recovery campaign (`replay.ordered-preload-recovery`, fixed and random) starts every history during the initial preload over on-demand markers and declares two more states: lazy demand in a replay that fails before the first ready, and a window move that waits behind a replay a newer truncate discards. A builder whose deferred move a discarded replay rejects, and one that keeps a failure before the first ready non-fatal, each failed only a named replay before; both now fail the fixed campaign, whose seed kills both (each dies in about half of 200-run seeds, so the random campaign kills each about half the time). Building the law found three model errors, each corrected from ARCHITECTURE.md: a failure is scoped to its logical demand, so a retired failed demand and the demand that replaces it publish when the replacement succeeds; a replay completes when its last participant's rows apply, so demand that a deferred window move starts afterwards does not join it; and an ordered tie request (a `where` on rank) is not lazy demand, which the terminal-error law had treated it as, so a failed tie request had excused a terminal error. A query's subscriptions to one collection now share one replay barrier (ARCHITECTURE.md, "Replay participants and failure"): a probe found that in a query that joins and includes the marker source, a marker truncate retired the include's only demand privately, its emptied replay closed while the join's replay still loaded, and include demand the graph then started failed outside every replay and ended the query. A sibling-demand branch of the failed-replay campaign declares that state; the branch's code before the fix fails the named replay and both campaigns at `convergence: terminal error`, and removing either sibling hold (at replay completion, or for an emptied replay) fails all three. The random failed-replay campaign then found the other half: one subscription's replay failed, the other's finished once it settled, and demand that a rank move then started on the finished subscription failed and ended the query, because the subscriber judged the failure by its own subscription. The subscriber now asks whether a replay of the source's collection failed; a named replay (`a demand that fails after the sibling subscription's replay failed`), a `failedSiblingDemandFailures` reach in the sibling branch (which now may move a row and fail the oldest held load), and both campaigns at 400 runs fail at `convergence: terminal error` on the per-subscription rule. A design that judged a failure against any failed replay of the query, across collections, used to survive, because the convergence check runs after an explicit window move, which retries a failed row demand. A strict terminal-error law now requires a failed lazy demand outside every recovery to end the query (ARCHITECTURE.md, error 228), skipping only failures the model cannot place: after a load threw, and, with a peer, a load that joined a replay. Merge with #2257: its review (CF01, [record](oracle-reviews/2026-10-26-pr-2257.md)) judged a new keyed request that a truncate commit starts as terminal, and called non-fatal recovery of that demand a future contract. This PR adopts that contract by maintainer decision. The merged owner keeps CF01's initiating-request receipt check as a second account of every terminal error, at each history cut and at convergence, plus its named replays and the `replay.subscription-demand-origin` and `replay.keyed-request-failures` campaigns, with their two reach-state pairs measured on this owner's model. Both demand origins, new demand during a marker commit and a settled demand that the replay asks for again, must stay nonterminal after the first ready. An always-fatal `failDemand` mutant fails 15 tests, including both ported campaigns; a never-fatal mutant fails the strict law in 8. CF01's duplicate of the #2224 named replay is folded into this owner's existing one. Merge with #2168 (ARCHITECTURE.md law 16): after initial-query readiness no load failure ends a live query, and a failed lazy demand is a request error on the query's node. That supersedes the strict terminal-error law, the error-228 rule, and the always-fatal and never-fatal mutant results above for a query that has been ready. The replay owner now allows a terminal error only after a load failed while the query was still loading, judged at each history cut and at convergence, and it adopts #2168's request-error witness (`requestErrors`); the excusals for thrown and unplaced failures went with the strict law. #2168's design already starts new demand before it releases retired coverage, and it treats a synchronous `loadSubset` throw as a rejection whose owner's release unloads it (the subscription oracles pin that unload), so the adapter-release law now counts a thrown load as acquired. It lacked the shared barrier, the failure hold behind a sibling's failed replay, and the deferred window move's wait, which the merge ports onto it. Without the deferred-move loop, 8 tests fail, #2221 trace 2 among them; without `failReplacement`, the two-failures named replay fails; without the sibling wait, that replay and both failed-replay campaigns fail. A deferred move rejects when a reload ends the sync run of the query or of a source, which the source-health oracle requires. The window-request reset went: a move can no longer throw, and deleting it failed nothing. The replay that a truncate opened without demand went too: under law 16 its demand's failure no longer ends the query, and its named replay passes on #2168's subscription. Review of the merge (2026-10-28): the replay owner gained a request-error account. After initial-query readiness, each failed lazy demand load that is still owed must appear among the query node's open request failures, by its injected error object, at every history cut, and no request error may stay after recovery replays every source (authority: ARCHITECTURE.md law 16, law 17, and the glossary's request error). A failure is met when its demand retires, a later load of the source covers its keys, or a later replay of the source completes; a failure inside a replay owes nothing until every participant settled. A terminal error now needs a load that failed while that same query was loading, so a peer's initial failure no longer excuses `live`. A thrown load joins a replay that no participant joined yet, and the failed-sibling branch now generates a held retry. `requestErrorChecks` reaches the declared minimum of 5 in each failed-replay campaign. Mutants: demand outcomes that never report an error fail 9 tests at the per-cut check; a successful replay that keeps its request errors fails 22 at the stale-error check. Limits and open items: the account shares one replay across a query's subscriptions to a source, so it cannot judge one subscription's request error while another load of that source is held. The reviewer's finding that `failReplacement`'s replay, which has no recorded failure, completes when every demand that joined it settles is closed as a law revision by maintainer decision: the hold ends when every demand load that joins it loads, a later replay of the source succeeds, or the subscription's last demand retires (ARCHITECTURE.md, "Replay participants and failure"). The query retries the failed demand at once and that retry joins the hold, so the hold cannot publish without the failed demand's rows. The replay oracle judges each subscription's request-error verdict by the subscription that a load names, and its failed-replay campaigns declare the premise (a subscription's load succeeds while that subscription's failure is still owed; 30 fixed and 24 random histories at the normal budget). No live-query history violates the request-error or retention law there; a mutant that completes such a hold when any participant settles fails both failed-replay campaigns at the retention law. A subscription-level `it.fails` test that asserted the earlier, stricter wording (no joined demand may end the hold) was removed with the revision. Limit: the oracle drives the hold only through live-query histories, where the retry always joins; a direct `Collection.subscribeChanges` caller that never retries is outside the live-query contract. Which failure a deferred window move rejects with, when its replay has several, is not specified: `main` rejected with the replay's earliest failure, and this branch rejects with `lastSubsetError`; by maintainer decision the move rejects with its replay's earliest failure that still decides it (law 17 decides which count: a retired demand's, a retried request's, and a replaced demand's failures do not). An identity law now judges deferred moves that start before any failure of the replays they wait on; on the parent code both failed-replay campaigns and the named replay `a window move behind a replay with two failed loads` fail with "replay error identity", and the deferred-move loop that rethrows a non-abort replay error passes. The request-error account decides failures per key: a later load of the source that asks for some of a failure's keys decides them, and keys it no longer asks for left the demand. The held-retry branch then found, at three times the run budget, a marker change publishing over a window that the model retained after a failed marker replay (`hold, truncateMarkers, settle ok, settle ok, settle fail newest, mark 3 off`, ascending, offset 1, limit 1, includes over on-demand markers). It was a model defect: the query retries a failed demand at once, and when that retry loads the failed keys the failure no longer counts (ARCHITECTURE.md law 17), so the replay succeeds and publishes. The model now clears a replay failure when a later load of the replay loads its keys. The replay oracle passes at the normal budget and three times at three times it, and production without `failReplacement` still fails its named replay at the retention law. A move whose `setWindow` throws synchronously after the request generation increments is reachable only through invariant errors, which probes for a throwing subscriber, a crashed source, and a throwing `select` did not reach. An aborted request's rejection and the failure of a segment that newer coverage released are not failures. That design now fails both failed-replay campaigns. An independent oracle review then narrowed several judgments. The terminal-error excuse after a synchronous throw applies only to the thrown error itself, ending the query after the step that threw; the throw's own step, and every other failure, are judged strictly (a per-history flag had excused every later failure). A load that fails outside every barrier after a failed demand of its source retired is unplaced, because the failed replay may still hold through the subscription's other demands, which the adapter cannot attribute. Limit: with a peer, the strict law accepts either query ending, since the adapter boundary cannot tell which query's subscription issued a load; attributing each failure to its query's subscription would close that. The window check now also requires every shown row to be eligible and to appear once, so a tie cannot hide a row the join rejects. A synchronous-throw dimension (`markerThrows` arms the next marker load to throw while it starts) found that a load throwing inside an open replay ended the query: the subscription dropped the demand, so its failure did not fail the replay, and the subscriber treated the throw as fatal. The subscription now hands the replay a rejected participant for such a throw and keeps the demand, which a later replay retries; named replays (`a replay load that throws while it starts`, `a demand whose load throws inside its open replay`), a `thrownReplayDemands` reach in a thrown-demand branch, and both failed-replay campaigns (now 600 runs) fail on the parent code, and a mutant that later unloads the never-acquired load fails the adapter-release check. Model corrections, each from the adapter-boundary rule that a thrown load acquires nothing: a thrown load joins a replay only if it still has a participant or a failure; an armed throw expires before the drain; and a thrown demand's retirement is invisible, so the retention law skips the marker barrier from a throw to the end of its episode (a later barrier in the episode inherits the ambiguity, which also corrects the existing release-throw case), and a failure that joins an already failed or ambiguous barrier may end the query. Maintainer decision: a synchronous throw inside an open replay fails that replay as a rejection does, so the subscription replay oracle's `new request throws` case and five `live-query-collection` cases, which expected a thrown new demand to roll back, now expect the rejection outcome. Limits: a design that dropped a thrown demand and let its replay succeed passes wherever only retention could see it; after a history command follows a throw, a terminal error is excused, because the model cannot see whether that command retired the thrown demand and with it its failed replay (a demand that waited on that replay then fails outside it). The named replays judge the throw's own outcome strictly. The failed-replay random campaign at five times its run count then found a retention counterexample with no throw, on the parent code too: a join and an include of one marker source; one subscription's replay fails, the other's finishes, and a row move starts demand on it whose load fails, which the shared barrier kept from ending the query; a marker change then retired the first replay's failed demand, which discarded that replay and published an empty window, because the second failure held nothing. That failure now fails its own subscription's replacement, which holds until a later replay succeeds or the subscription's last demand retires. A named replay of the trace fails without the hold, and a `partialFailureRetirements` reach (a failed demand retires while another failure still holds the barrier) is declared in the failed-replay campaign; the campaigns themselves do not fail without the hold, so the named replay is the witness. A window move whose load throws while it starts counts as a settled, failed move in the model (it used to stay pending, so the newest-move law misjudged the older move), with a `thrownMoves` reach. The broad random campaign at five times its run count then found an older move whose settlement resolved with itself: the newer move threw after it bumped the request generation and before it replaced the latest request, so the older move's settlement returned its own promise, a chaining cycle that escaped as a TypeError. A call now clears the latest request when it bumps the generation, so no older move can wait on itself; a named replay fails without that. The same assignment covers a move deferred behind a replay, which had the same shape (an independent review found it can hang forever when a newer move throws between the replay settling and the deferred move resuming); that microtask gap is outside the grammar, which flushes between commands, so only the shared mechanism, not that path, has a witness. The ordered-work oracle also keeps the trace from #2221 as a named replay: before prefix growth the loader read the full source twice after a signalling refetch past a NaN boundary; with growth it reads none. Limits: the adapter boundary cannot tell which query issued a load, so the retention law and the participant rule skip histories with a peer. The retention law judges barriers that rise after the query first became ready; before that the initial-readiness law governs. It skips a barrier where a release threw while the replay started, which may fail that replay. Core now reports a throwing `unloadSubset` uncaught and fails nothing with it; the subscription lifecycle, replay, Effect disposal, includes-temporal, and subset-error oracles check the uncaught report, once, with the adapter's error. The request-authority oracle now also generates requests that throw while they start, the last history only TLA+ covered, with an error-identity law: a failed request's participant reports that request's own error. It found that a superseded failure whose recovery read threw at its start reported the start error instead (review finding 6). A cost law then checks, after loading is quiet and a bounded repair is available, that one invalidation reads no full source and issues at most one chain per ordering epoch move (in the fake, a prefix, a tie, and a refill); a mutant that always repairs with a full-source read fails it. The replay-refinement section's status law requires a quiet live query and its peer to stay ready when no load failed. Limits: whether a release runs the graph at once or defers it to a transaction belongs to the scheduler, and the oracle generates both forms without checking the scheduler's choice; the fake's rows do not move except for the unexpressible boundary; publication holds stay with other tests; the laws do not judge requests issued before a truncate; a hung authoritative request suspends the leak law for its unfinished chain; a recovery start that throws during a superseded failure is rare in the campaigns, so a named replay owns it. | +| Ordered acquisition | Complete | Exact demand identity, applied settlement, independent pagination recomputation, request work, lifecycle products, replay authority, source-generation readiness, and transaction-refinement abort boundaries are literate. An initial full-source ordered load shares the synchronous readiness cut ([historical review](oracle-reviews/initial-full-source-sync-readiness.md)). The ordered-lifecycle oracle covers an inexpressible-boundary fallback before first publication when every acquisition returns literal `true`. It also checks a later null-boundary fallback after the first joined window publishes, across literal, Promise, and held-receipt responses, including a held receipt after child demand settles ([review](oracle-reviews/issue-2152-initial-ordered-fallback.md)). The ordered loader state oracle also has a generated request-authority lane (`ordered-loader.request-authority`, fixed and random campaigns). It drives the real loader over a fake subscription through settlements in any order, failures, deferred invalidation and report reactions, ordinary passes and window moves, and began with four laws from a TLA+ model (#2191): a successful repair keeps its acquisitions until a covering read succeeds; quiet loading holds nothing older than its last authoritative read; a superseded request's failure is recovered by a full-source read without an explicit retry; and once quiet, an invalidation and a pass issue a read unless a full-source read covers the source. Scenarios also throw from the first release callback, answer a repair synchronously from installed rows, and let a page's own delivery invalidate ordering; named replays keep the traces they found, each of which failed on main (a throwing release that unwound a resumed repair, a synchronous repair waiting behind a superseded request, and a self-invalidating page whose tie was never retired). On main it found a superseded tie that failed after its repair finished and a newer invalidation's reaction was deferred, which left refills blocked; a named replay keeps that trace. A stale prefix that retires the repair's prefix fails the held law, and a stale request held after the repair finished fails the leak law. The lane also crosses unexpressible boundaries, joined work that defers loading, truncate replay that succeeds or stays pending, optimistic departures, source deletes, the bounded backoff retry, window moves that shrink or widen, plans that read only the full source, and reports inside the loader's own first page. A replay owns the requests issued before its truncate, and when the subscription still holds a full-source lease the settled replay covers the source. These dimensions found three more defects, each kept as a named replay: a repair prefix that settled after a replay started its replacement took the replacement's chain tag and was never retired; a replay counted a full-source read as complete after an explicit retry had released it; and a failure from before a truncate dropped the next repair's staged step. Each of the seven defect replays fails on the loader at the branch's merge base with main. A later law, one request at a time, found that a replaced repair chain's prefix still counted as current and issued a tie beside the replacement's prefix; a chain is now superseded by any newer authority start. The failed-lease law also counts a covering replay, which a named replay reaches. Repeated random campaigns then found a repair request that settled after full-source coverage and stayed held; full coverage now retires it. The held law counts a later prefix that a successful replay reacquired while it was in flight. Four histories from TCP's known error classes followed: a request the provider never answers (a lost segment), a release callback that truncates synchronously (a reentrant reset), a window offset other than zero, and an at-most-once release law with no release while a replay owns the demand unless an explicit retry replaces it or a held full-source lease covers it. They found that the backoff retry's release pass kept releasing failed demands after a release started a truncate; every release pass now stops on a reentrant reset. The oracle now observes epoch moves as calls to `invalidateSourceOrdering()`, so it tells a replacement chain from a chain step exactly. The loader's work slot and debt now move only through a transition table that throws `OrderedLoaderInvariantError` on an illegal move or an incoherent work and debt pair; the oracle records any such error as a broken law, so its generated histories fuzz the table. A covering-lease law (a full-source lease whose read succeeded or whose demand a replay reacquired is never released) found that a full-source transport failing after a covering replay turned complete coverage into an own failure; the table then found that a repair request failing after a replay finished its chain did the same. Mutants that drop each new guard fail. A release callback that runs an ordinary pass synchronously, as a graph run does when the release delivers deletes outside a transaction, found that the pass issued a page while another request was settling, and that request then issued its tie; every release pass now blocks refinement until it unwinds. A dispose command and a release callback that disposes the loader check that a disposed loader issues and releases nothing; mutants that drop the loader's own `active` checks in the release pass and in failure handling survive because dispose resets the cursor, so those checks were removed. TLA+ run 3 (`LoaderTable.tla`) models the table with truncate and replay; its safe rules pass with 9,368,122 distinct states, and each of the four guards from this round, removed alone, produces a counterexample. Reverting any one fix fails its replay and a campaign. The generated histories also stack a second truncate on a pending replay and truncate after a failed replay, each with a reach witness. A lease missing from its map during a release pass is now an invariant violation; no history reaches it. A second high review found that a covering replay did not retire a superseded failed lease from before its truncate, because the failed-lease law exempted every replay-owned request. The law now judges a superseded failure that was live (it failed in the incarnation it was issued in) once a covering replay settles after it, and the replay retires those leases. The subscriber now schedules the replay's graph run in a finally. The replay-refinement oracle is the receiving witness: its generated ordered section (`replay.ordered-window-faults`) drives a real on-demand Collection and an ordered live query through rank moves, held and failed loads, truncates, and unloads that throw without a truncate, while a replay starts, and while it settles. Its settled-window law found that a release throwing after a successful ordered load left the window on old rows, because the builder counted the rejected participant as a failed load (also on main); the loader now marks release errors and the builder publishes. It also fails without the subscriber's finally. The section now also crosses tied ranks (its law compares the window's rank sequence and each shown row's current rank), a peer live query on the same source, offset 1, descending order, and explicit window moves; a move that rejects after a failed source recovery leaves the window, and one rejected only by a release failure still moves. A reported-faults law accepts injected faults as rejected loads, thrown releases, rejected moves, or errors the subscription rethrows from a microtask, and rejects any other escaped error. These found that a window move whose load succeeded was recorded as failed when a release threw, so the window stayed private until another move. A throwing release is an adapter bug, so the loader now reports it uncaught from a microtask and changes nothing else; the reported-faults law rejects a move that a release failure rejects, and the request-authority oracle requires each throwing release to be reported uncaught exactly once (a mutant that swallows the error fails). It also crosses a joined marker source: a LEFT join that keeps marked or unmarked rows, and an inner join. The marker source is eager or on-demand, with marker loads that are held, fail, and truncate. This found that an on-demand joined source whose loads finished without delivering a row left the window on old rows, because nothing ran the graph that the pending joined work held; the subscriber now runs it when a joined source becomes ready. A truncate never aborts a window move: the move waits for any replay that holds publication and then applies (ARCHITECTURE.md), so a move that rejects with `AbortError` during a history is an escaped error. (An earlier version of this entry claimed a named replay reached an aborted-move branch; no history reached it, and the branch, which quietly accepted such a rejection, is gone.) Recovery truncates the marker source too, because a failed marker replay keeps the last complete publication until that source replays. A hostile-mutant pass over the loader's guards found guards that no ordered oracle kills. A differential instrument then ran 3,000 generated request-authority histories through the loader and each mutant and compared every request's shape, outcome, and release. Three guards the code makes redundant are deleted: the backoff timer guard, the explicit-move check in the retry callback, and the `requiresFullSource` check on a report. An ignored-report law now kills two of the mutants that changed public requests: a report issues no authoritative request before any request settled (a request in flight still owes an exact result for its own options) or while the window is inside the loader's own first prefix. Named replays reach both, and the random campaigns judge them in quiet histories. A random ordered-work campaign later found that the first-prefix check was unsound: it took the next installed row as the next source row, but a signalling refetch moved the page's end before the tie the loader had read, so that tie's row stood past the page while a row tied with the new end was never sent, and a later window move skipped it. The loader now accepts the row only when its page read it or when it read the tie of the page's current end. When the end moved, it repairs (see the later maintainer decision below); query-db's first-page witness requires bounded reads only, no full-source read, and an unchanged count of active Queries. The optimistic-window cases that allow but do not require a refill still bound any refill that occurs: an ordered read within the refill bound or a predicate read, never the full source (a mutant that made the growth read unbounded fails them). The ordered-work model now sorts `undefined` ranks with `null` ones, as the comparator does, and its histories generate both; nullish ranks tie in its window and in-flight comparisons. An `undefined` end also matches a loader that attempted no tie; a nullish end is never an expressible cursor, so the loader has started or will start a full-source read, which covers the source either way. A guard against that match survived the generated campaigns as an equivalent mutant and was removed. The ordered-work oracle keeps the trace as a named replay that fails on main, and the request-authority oracle has an unproven-report law (a report over a first page that no row it read proves starts a repair or reads the end's tie), whose replay and fixed campaign fail on main; the broad campaigns reach that law in about one history in 10,000, so an unproven-report campaign (`ordered-loader.unproven-report`, fixed and random) settles the first page, settles the read after it, and reports, over random case parameters and a random suffix, and declares its reach (about a third of 300 histories); `main`'s loader fails both runs. Its fake subscription now reads installed rows as the subscription does, applying a tie's equality, a cursor, and a limit; when its reads ignored the limit, the tie was always the last installed row's, so the first-prefix ignore branch was unreachable. The cost law's per-chain bound, three requests, was an artifact of that fake: it is now a prefix plus a tie and a page per installed row, and it also judges an unindexed window that still needs rows, which grows its prefix instead of reading the full source. The report campaign (`ordered-loader.unproven-report`, 1,000 runs; at 600 a random seed could miss the minimum) now starts each history with one of eight prefixes and declares reach for each report premise: a proven first page, a page that read past a window that shrank, a held full-source read, a head insert that moved the page's end, an owed repair, authoritative work in flight, a release of the older tie that truncates or disposes the loader, and a report to a disposed loader. The unproven-report law requires a bounded repair or the end's tie when every installed rank is expressible; a full-source read no longer satisfies it (a mutant that forced full debt after a report passed before). The cost law also rejects a repeated read: within one invalidation, no prefix, tie, or page repeats more often than the ordering epoch moved. Two laws are stronger: an ignored report, over a proven first page or a held full-source read, issues no request at all, where it used to forbid only authoritative ones; and a report while a repair is owed reads no tie or page before the repair. Deleting the moved-end early return in `revalidateLoadedCoverage`, or its past-window term, fails the first law, and deleting the owed-debt term in `loadMovedEndTie` fails the second; a deletion sweep had found all three surviving every oracle. With that reach, the report path's own `!active` checks were deleted: `dispose()` advances the generation that the stale-run check after the release reads, and a disposed loader issues nothing, which the dispose law judges. A head insert can carry a `-Infinity` rank, and two more laws judge every request: the loader never reads the tie of a rank no cursor can express, and the first finite request after a truncate has no cursor. The fake's `dataNeeded` now counts the window rows not at or before the boundary it is given, where it returned a constant. With those, the stale-run check after the older tie's release fails the reset law when a release truncates the loader and the window then grows (a prefix declares that reach), and removing `loadBoundary`'s own unexpressible-value check fails the unexpressible-tie law. The covered-source early return and the staged-continuation term were deleted: coverage that a full-source read completed holds the work slot, whose check then decides the report, and a staged finite continuation cannot coexist with a proven end, while a staged repair step owes a repair. The ignored-report law's full-source branch no longer needs a quiet history, since such coverage owes nothing, and a prefix reaches a paged loader whose repair fell back to a full-source read; removing the work-slot check then fails it. The unexpressible-end term in `loadMovedEndTie` stays: the unindexed-refetch random campaign found that without it an unindexed loader reads the full source through the boundary fallback where its bounded prefix repair would do, and a named replay of that trace fails without the term. Maintainer decision, later: a report whose page end moved now repairs instead of reading only the new end's tie, to cut shipped bytes; `loadMovedEndTie` and its guards are gone, so their mutants above no longer exist. The laws stay and judge the repair path: the ignored-report, owed-repair, unexpressible-tie, and reset laws all still run, and the unproven-report law accepts a repair. The report campaign drops its two prefixes whose release of the older tie truncated or disposed the loader, since a report no longer releases anything, and the query-db replay of an insert into a top-N list now expects the bounded repair's prefix and the new end's tie. Still open there: the `Object.is` term of the moved-end check. It skips work when the loader already read the end's tie, and it decides differently from its fallback only while other work is in flight, which the ignored-report law's quiet premise excludes. The ordered-work oracle owns that branch instead: peers over a refetching provider stay in its grammar for source filters (#2155 excludes only joined filters), and a proven-report law requires that a report which changes nothing the first query's own page and end tie prove, with the model's next row installed and no installed row deleted or reordered, starts no request for it and leaves it owing no `refetch: true` repair. A campaign fixed to that premise must reach it, and a loader that repairs every report fails both its fixed and random runs. A staged end tie that pending joined work defers is read by the pass after that work finishes; a request-authority named replay reaches it, and a loader that drops the staged tie fails it. The unproven-report law judges that later pass, and a truncate, failure, move, or disposal in between voids what the report owed. A history that starts during the initial preload found that a window move deferred behind a marker replay, which the source truncate then discarded, rejected with that replay's `AbortError`, so the older move's window won (also on main). A deferred move now waits for every replay that holds publication and applies unless one failed or cleanup ended the sync run; a named replay fails without the fix. The ordered-work refetch law also required no full-source read after a signalling refetch for an unindexed plan, which the architecture then let refill a short window from the full source; random seeds found it on main. Maintainer decision on this branch: an underfilled unindexed prefix never reads the full source, because that read is unbounded. It grows instead: each growth asks for twice the last prefix, and it stops once the last growth added no rows. A deletion sweep showed that the older condition's other term, a prefix that reads short, follows from that one. The refetch law now forbids a full-source read after a signalling refetch with no unindexed exception. The named replay of a short unindexed window and the fixed unindexed-refetch campaign (`ordered-work.eager-unindexed-refetch`, fixed and random, with reach on windows the prefix fills and windows it leaves short) failed on the full-source fallback and pass with growth; the full-source mutant still fails them. Growth made a new optimistic history possible: an unindexed one-row window behind a joined or anti-joined filter that installs only the prefix it grew to, whose departure leaves the replacement row missing. Sixteen fixed optimistic-window cases (update and delete, both consumers, acknowledge and rollback) cover it, with a refill bound of twice the installed prefix plus the overlaid row. A loader without the stop rule never reaches a fixed point; a loader that stops whenever the prefix reads short leaves those windows empty after settlement, because the pending departure hides a row from the count. The existing unindexed optimistic cases now allow, but do not require, a request behind a joined filter when the replacement row is installed: growth proves rows only up to its prefix, where the full-source fallback had proven every row (a maintainer-approved expectation change). The proven-report random campaign now declares its reach too. One mutant still changes requests without a law: dropping the optimistic-refill flag turns a refill into a refetching prefix. Six more are equivalent in the fake but stay, because the fake cannot reach their premise or they encode a documented rule: the own-first-page check behind a held sync transaction (the fake never holds one), a truncate moving complete coverage back to held, clearing the pending request when a full-source read starts (the fake never answers a full-source read synchronously), staging a joined `more` continuation, the staged-continuation guard before an unindexed prefix, and the drain re-entry guard (a long synchronous chain would otherwise recurse). An include dimension adds a `toArray` child that reads the marker source for each shown row, and an include-children law requires each shown row's markers to equal the model wherever the window law holds; a mutant that never loads newly demanded child keys fails it in an include-only scenario. A newest-move law requires a superseded window move to resolve only after the newest move settles (a superseded move that fails still rejects with its own error), and the model lets the newest move that resolved set the window; it found that a superseded move resolved as soon as its own load settled, so awaiting it showed neither window. The terminal-error law also allows `error` after a load fails while the query is still loading, the documented initial-load failure; a lazy join source is exempt. Limits: includes are one level deep. It also found that a release failure in lazy joined demand during a window move rejected the move, because the subscription reported it as a load error. A window law now lets the newest move win; it found that two moves deferred by a pending source recovery could resume in the opposite order, so the older request replaced the newer window. `setWindow()` now ignores a deferred call that a newer call superseded. A terminal-error law allows a final `error` only after a lazy join demand load, on either side, failed outside a truncate's recovery; named replays reach that branch. A load belongs to a recovery when it asks again for a demand its source held before the truncate (the same request data), or when the graph starts it while the truncate commits. The first model read a flag set only during the commit, but subscriptions start replay loads in a later microtask, so it counted every replay load as new demand and allowed `error` after any failed replay load; a random campaign found the reverse case (#2224): a marker truncate privately moved the window, the include's new demand started outside any replay because releasing its old demand ended the subscription's replay, and its failure ended the query. The demand controller now starts new coverage before it releases retired coverage, a query subscription opens a replay at every truncate even without demand (a replay that starts no recovery work and reports no loading), and a demand failure that fails its subscription's replay keeps the last publication instead of ending the query once the query has first become ready. The section also generates histories that start during the initial preload, with an initial-readiness law (the preload has settled once loading is quiet), marker loads whose aborted requests reject with `AbortError` as the source contract asks, and settles of the newest held load. These found two older defects on main: an aborted lazy demand request that rejected after its truncate replay succeeded ended the query, and so did a superseded segment that the controller released; the controller now treats an aborted request as obsolete demand, which its replay or newer coverage decides. Named replays keep each trace; reverting each production change fails one of them, and a mutant whose replay rebuilds its request data fails the campaigns' replay-load reach witness. The ordered-lifecycle oracle owns the truncate-work witness: a zero-limit window issues no request and reports no loading at a truncate, while a one-row window reacquires. A retention law then judges law 9 at generated checkpoints: a truncate raises a publication barrier, every load its source starts while the barrier is up joins it, a participant's failure fails the replay while its logical demand is held, and while any barrier is up the query shows exactly the rows it showed when the first one rose. It runs after every command and once loading is quiet. A failed-replay campaign (`replay.ordered-failed-replay`, fixed and random) starts each history by holding loads, truncating a source, optionally starting demand, and failing a held load, with a branch that fails an include demand while its marker replay still loads; it declares four reach states with `createOracleReach` (a judged failed replay, a judged held replay, a demand started after the commit that fails while its replay loads, and the include form of that demand). A replay participant now also counts as recovery for the terminal-error law, so a demand that starts after the commit while its replay is open is judged in generated histories, not only by a named replay. A mutant whose failed replay releases the publication gate fails both failed-replay campaigns and the broad campaigns; a mutant that ends the query when such a demand fails fails both failed-replay campaigns; the branch's production files reverted to `main` fail both. A preload-recovery campaign (`replay.ordered-preload-recovery`, fixed and random) starts every history during the initial preload over on-demand markers and declares two more states: lazy demand in a replay that fails before the first ready, and a window move that waits behind a replay a newer truncate discards. A builder whose deferred move a discarded replay rejects, and one that keeps a failure before the first ready non-fatal, each failed only a named replay before; both now fail the fixed campaign, whose seed kills both (each dies in about half of 200-run seeds, so the random campaign kills each about half the time). Building the law found three model errors, each corrected from ARCHITECTURE.md: a failure is scoped to its logical demand, so a retired failed demand and the demand that replaces it publish when the replacement succeeds; a replay completes when its last participant's rows apply, so demand that a deferred window move starts afterwards does not join it; and an ordered tie request (a `where` on rank) is not lazy demand, which the terminal-error law had treated it as, so a failed tie request had excused a terminal error. A query's subscriptions to one collection now share one replay barrier (ARCHITECTURE.md, "Replay participants and failure"): a probe found that in a query that joins and includes the marker source, a marker truncate retired the include's only demand privately, its emptied replay closed while the join's replay still loaded, and include demand the graph then started failed outside every replay and ended the query. A sibling-demand branch of the failed-replay campaign declares that state; the branch's code before the fix fails the named replay and both campaigns at `convergence: terminal error`, and removing either sibling hold (at replay completion, or for an emptied replay) fails all three. The random failed-replay campaign then found the other half: one subscription's replay failed, the other's finished once it settled, and demand that a rank move then started on the finished subscription failed and ended the query, because the subscriber judged the failure by its own subscription. The subscriber now asks whether a replay of the source's collection failed; a named replay (`a demand that fails after the sibling subscription's replay failed`), a `failedSiblingDemandFailures` reach in the sibling branch (which now may move a row and fail the oldest held load), and both campaigns at 400 runs fail at `convergence: terminal error` on the per-subscription rule. A design that judged a failure against any failed replay of the query, across collections, used to survive, because the convergence check runs after an explicit window move, which retries a failed row demand. A strict terminal-error law now requires a failed lazy demand outside every recovery to end the query (ARCHITECTURE.md, error 228), skipping only failures the model cannot place: after a load threw, and, with a peer, a load that joined a replay. Merge with #2257: its review (CF01, [record](oracle-reviews/2026-10-26-pr-2257.md)) judged a new keyed request that a truncate commit starts as terminal, and called non-fatal recovery of that demand a future contract. This PR adopts that contract by maintainer decision. The merged owner keeps CF01's initiating-request receipt check as a second account of every terminal error, at each history cut and at convergence, plus its named replays and the `replay.subscription-demand-origin` and `replay.keyed-request-failures` campaigns, with their two reach-state pairs measured on this owner's model. Both demand origins, new demand during a marker commit and a settled demand that the replay asks for again, must stay nonterminal after the first ready. An always-fatal `failDemand` mutant fails 15 tests, including both ported campaigns; a never-fatal mutant fails the strict law in 8. CF01's duplicate of the #2224 named replay is folded into this owner's existing one. Merge with #2168 (ARCHITECTURE.md law 16): after initial-query readiness no load failure ends a live query, and a failed lazy demand is a request error on the query's node. That supersedes the strict terminal-error law, the error-228 rule, and the always-fatal and never-fatal mutant results above for a query that has been ready. The replay owner now allows a terminal error only after a load failed while the query was still loading, judged at each history cut and at convergence, and it adopts #2168's request-error witness (`requestErrors`); the excusals for thrown and unplaced failures went with the strict law. #2168's design already starts new demand before it releases retired coverage, and it treats a synchronous `loadSubset` throw as a rejection whose owner's release unloads it (the subscription oracles pin that unload), so the adapter-release law now counts a thrown load as acquired. It lacked the shared barrier, the failure hold behind a sibling's failed replay, and the deferred window move's wait, which the merge ports onto it. Without the deferred-move loop, 8 tests fail, #2221 trace 2 among them; without `failReplacement`, the two-failures named replay fails; without the sibling wait, that replay and both failed-replay campaigns fail. A deferred move rejects when a reload ends the sync run of the query or of a source, which the source-health oracle requires. The window-request reset went: a move can no longer throw, and deleting it failed nothing. The replay that a truncate opened without demand went too: under law 16 its demand's failure no longer ends the query, and its named replay passes on #2168's subscription. Review of the merge (2026-10-28): the replay owner gained a request-error account. After initial-query readiness, each failed lazy demand load that is still owed must appear among the query node's open request failures, by its injected error object, at every history cut, and no request error may stay after recovery replays every source (authority: ARCHITECTURE.md law 16, law 17, and the glossary's request error). A failure is met when its demand retires, a later load of the source covers its keys, or a later replay of the source completes; a failure inside a replay owes nothing until every participant settled. A terminal error now needs a load that failed while that same query was loading, so a peer's initial failure no longer excuses `live`. A thrown load joins a replay that no participant joined yet, and the failed-sibling branch now generates a held retry. `requestErrorChecks` reaches the declared minimum of 5 in each failed-replay campaign. Mutants: demand outcomes that never report an error fail 9 tests at the per-cut check; a successful replay that keeps its request errors fails 22 at the stale-error check. Limits and open items: the account shares one replay across a query's subscriptions to a source, so it cannot judge one subscription's request error while another load of that source is held. The reviewer's finding that `failReplacement`'s replay, which has no recorded failure, completes when every demand that joined it settles is closed as a law revision by maintainer decision: the hold ends when every demand load that joins it loads, a later replay of the source succeeds, or the subscription's last demand retires (ARCHITECTURE.md, "Replay participants and failure"). The query retries the failed demand at once and that retry joins the hold, so the hold cannot publish without the failed demand's rows. The replay oracle judges each subscription's request-error verdict by the subscription that a load names, and its failed-replay campaigns declare the premise (a subscription's load succeeds while that subscription's failure is still owed; 30 fixed and 24 random histories at the normal budget). No live-query history violates the request-error or retention law there; a mutant that completes such a hold when any participant settles fails both failed-replay campaigns at the retention law. A subscription-level `it.fails` test that asserted the earlier, stricter wording (no joined demand may end the hold) was removed with the revision. Limit: the oracle drives the hold only through live-query histories, where the retry always joins; a direct `Collection.subscribeChanges` caller that never retries is outside the live-query contract. Which failure a deferred window move rejects with, when its replay has several, is not specified: `main` rejected with the replay's earliest failure, and this branch rejects with `lastSubsetError`; by maintainer decision the move rejects with its replay's earliest failure that still decides it (law 17 decides which count: a retired demand's, a retried request's, and a replaced demand's failures do not). An identity law now judges deferred moves that start before any failure of the replays they wait on; on the parent code both failed-replay campaigns and the named replay `a window move behind a replay with two failed loads` fail with "replay error identity", and the deferred-move loop that rethrows a non-abort replay error passes. The request-error account decides failures per key: a later load of the source that asks for some of a failure's keys decides them, and keys it no longer asks for left the demand. The held-retry branch then found, at three times the run budget, a marker change publishing over a window that the model retained after a failed marker replay (`hold, truncateMarkers, settle ok, settle ok, settle fail newest, mark 3 off`, ascending, offset 1, limit 1, includes over on-demand markers). It was a model defect: the query retries a failed demand at once, and when that retry loads the failed keys the failure no longer counts (ARCHITECTURE.md law 17), so the replay succeeds and publishes. The model now clears a replay failure when a later load of the replay loads its keys. The replay oracle passes at the normal budget and three times at three times it, and production without `failReplacement` still fails its named replay at the retention law. A move whose `setWindow` throws synchronously after the request generation increments is reachable only through invariant errors, which probes for a throwing subscriber, a crashed source, and a throwing `select` did not reach. An aborted request's rejection and the failure of a segment that newer coverage released are not failures. That design now fails both failed-replay campaigns. An independent oracle review then narrowed several judgments. The terminal-error excuse after a synchronous throw applies only to the thrown error itself, ending the query after the step that threw; the throw's own step, and every other failure, are judged strictly (a per-history flag had excused every later failure). A load that fails outside every barrier after a failed demand of its source retired is unplaced, because the failed replay may still hold through the subscription's other demands, which the adapter cannot attribute. Limit: with a peer, the strict law accepts either query ending, since the adapter boundary cannot tell which query's subscription issued a load; attributing each failure to its query's subscription would close that. The window check now also requires every shown row to be eligible and to appear once, so a tie cannot hide a row the join rejects. A synchronous-throw dimension (`markerThrows` arms the next marker load to throw while it starts) found that a load throwing inside an open replay ended the query: the subscription dropped the demand, so its failure did not fail the replay, and the subscriber treated the throw as fatal. The subscription now hands the replay a rejected participant for such a throw and keeps the demand, which a later replay retries; named replays (`a replay load that throws while it starts`, `a demand whose load throws inside its open replay`), a `thrownReplayDemands` reach in a thrown-demand branch, and both failed-replay campaigns (now 600 runs) fail on the parent code, and a mutant that later unloads the never-acquired load fails the adapter-release check. Model corrections, each from the adapter-boundary rule that a thrown load acquires nothing: a thrown load joins a replay only if it still has a participant or a failure; an armed throw expires before the drain; and a thrown demand's retirement is invisible, so the retention law skips the marker barrier from a throw to the end of its episode (a later barrier in the episode inherits the ambiguity, which also corrects the existing release-throw case), and a failure that joins an already failed or ambiguous barrier may end the query. Maintainer decision: a synchronous throw inside an open replay fails that replay as a rejection does, so the subscription replay oracle's `new request throws` case and five `live-query-collection` cases, which expected a thrown new demand to roll back, now expect the rejection outcome. Limits: a design that dropped a thrown demand and let its replay succeed passes wherever only retention could see it; after a history command follows a throw, a terminal error is excused, because the model cannot see whether that command retired the thrown demand and with it its failed replay (a demand that waited on that replay then fails outside it). The named replays judge the throw's own outcome strictly. The failed-replay random campaign at five times its run count then found a retention counterexample with no throw, on the parent code too: a join and an include of one marker source; one subscription's replay fails, the other's finishes, and a row move starts demand on it whose load fails, which the shared barrier kept from ending the query; a marker change then retired the first replay's failed demand, which discarded that replay and published an empty window, because the second failure held nothing. That failure now fails its own subscription's replacement, which holds until a later replay succeeds or the subscription's last demand retires. A named replay of the trace fails without the hold, and a `partialFailureRetirements` reach (a failed demand retires while another failure still holds the barrier) is declared in the failed-replay campaign; the campaigns themselves do not fail without the hold, so the named replay is the witness. A window move whose load throws while it starts counts as a settled, failed move in the model (it used to stay pending, so the newest-move law misjudged the older move), with a `thrownMoves` reach. The broad random campaign at five times its run count then found an older move whose settlement resolved with itself: the newer move threw after it bumped the request generation and before it replaced the latest request, so the older move's settlement returned its own promise, a chaining cycle that escaped as a TypeError. A call now clears the latest request when it bumps the generation, so no older move can wait on itself; a named replay fails without that. The same assignment covers a move deferred behind a replay, which had the same shape (an independent review found it can hang forever when a newer move throws between the replay settling and the deferred move resuming); that microtask gap is outside the grammar, which flushes between commands, so only the shared mechanism, not that path, has a witness. The ordered-work oracle also keeps the trace from #2221 as a named replay: before prefix growth the loader read the full source twice after a signalling refetch past a NaN boundary; with growth it reads none. Limits: the adapter boundary cannot tell which query issued a load, so the retention law and the participant rule skip histories with a peer. The retention law judges barriers that rise after the query first became ready; before that the initial-readiness law governs. It skips a barrier where a release threw while the replay started, which may fail that replay. Core now reports a throwing `unloadSubset` uncaught and fails nothing with it; the subscription lifecycle, replay, Effect disposal, includes-temporal, and subset-error oracles check the uncaught report, once, with the adapter's error. The request-authority oracle now also generates requests that throw while they start, the last history only TLA+ covered, with an error-identity law: a failed request's participant reports that request's own error. It found that a superseded failure whose recovery read threw at its start reported the start error instead (review finding 6). The state oracle compares every current failed request's participant reason with its own originating Error by identity. Release cleanup cannot replace that reason, even when the thrown value's message starts with “release ”. A bounded 16-case checker calibration crosses index availability, widths 1/2, actual throwing release absent/present, and ordinary/full-source plans. It feeds three wrong participant reasons into the same model/production-driver/comparison; the original prefix waiver and an actual-release-only waiver fail at that check. The existing fixed/random 10,000 authority histories declare and enforce current-request-error reach and actual cleanup-fault/current-request-error overlap. Final evaluation counts are 3888/1431 fixed and 3883/1388 random. The full owner gives 50/50 GREEN, preserving retirement 200 fixed/random controlled and real-subscription receiving histories. Scope ends at the loader participant and separate uncaught-cleanup observations; it adds no provider/framework or intermediate-publication promise. The state owner's shared error-identity calibration and paired generated campaigns enforce these checks. A cost law then checks, after loading is quiet and a bounded repair is available, that one invalidation reads no full source and issues at most one chain per ordering epoch move (in the fake, a prefix, a tie, and a refill); a mutant that always repairs with a full-source read fails it. The replay-refinement section's status law requires a quiet live query and its peer to stay ready when no load failed. Limits: whether a release runs the graph at once or defers it to a transaction belongs to the scheduler, and the oracle generates both forms without checking the scheduler's choice; the fake's rows do not move except for the unexpressible boundary; publication holds stay with other tests; the laws do not judge requests issued before a truncate; a hung authoritative request suspends the leak law for its unfinished chain; a recovery start that throws during a superseded failure is rare in the campaigns, so a named replay owns it. | | Indexed predicate filtering | Complete for bounded dotted-path collision | An independent row filter checks direct predicates and public callback predicates for a dotted scalar field beside a nested path, with and without an index. Selected-field ordering checks the same distinction through `$selected`. | | WHERE predicate publication | Complete for bounded predicate and sync-transaction grammar | The contract, Kleene reference evaluator, snapshot and change-history grammar, three subscriber consumers plus direct snapshot, and per-commit key-set refinement check are literate. A fixed same-key update checks live-row and direct-subscriber payloads; a focused descriptor boundary checks stored-row prefilter safety. Generated cleanup and restart histories for filtered subscribers remain open. | @@ -1780,7 +1780,7 @@ generic visitor behavior outside the named collectors. | Observer hydration under terminal statuses | `packages/db/tests/live-query-observer-hydration-oracle.property.test.ts` | ARCHITECTURE.md law 16 and the glossary "crash": while a hydration seed is active, a Collection `error` or `crashed` shows in the snapshot, attached or detached, and is announced; an attached observer that sees the failure ends the seed, so cleanup then shows `cleaned-up` and live rows, not the stale failure or the seed. Decision (not a law-16 claim): `cleaned-up` and restart under an active seed keep the seed as `ready`, because cleanup is not a failure and the next sync run's readiness hands off. Generated histories cross both modes, attach order, live writes, `markReady()`, `fail()`, `crash()`, cleanup, and restart, with declared reach; replays pin the crash, fail-then-cleanup, and crash-before-attach traces. Limits: rows under a failure status are not claimed; the source is a plain controlled Collection, not a live-query graph or an on-demand subset; streamed client errors, handoff timing, and framework adapters stay with the focused observer and adapter tests; the property is not in the `oracle-config.ts` path-replay registry. | | Query DB SSR dehydration | `packages/query-db-collection/tests/ssr-dehydration-oracle.test.ts`, [review record](oracle-reviews/issue-1950-ssr-dehydration.md) | A plain Query cache control, a live on-demand compound predicate, a signal-only request, and an ordered cursor with a custom comparator all retain their row through QueryClient dehydration, reconstruction of Seroval's emitted stream payload, and Query Core hydration from that payload. The on-demand query functions still receive the original IR, comparator, and signal; serialized metadata retains enumerable user metadata but excludes request options. The original implementation fails the compound and signal cases at the stream checkpoint. A serializable stream-only request-options leak passes the former chunk-count check but fails the current payload assertion. This owner covers successful Query cache entries at the installed Query Core 5.90.20 and Seroval 1.5.0 versions. A TanStack Start integration owner still needs the reporter's Router 1.171.33 and Seroval 1.6.8 path, browser Collection resume/refetch, and request cancellation across SSR. Query subset and pagination oracles remain owners of predicate/order/cursor meaning and supported value types. The issue's `shouldDehydrateQuery` workaround replaces Query Core's success-only default and admits pending/error queries; any documented workaround must preserve that default filter. | | Query DB ordered refetch | `packages/query-db-collection/tests/refetch-ordered-prefix-oracle.test.ts` | Issue #2136: after preload, and after `utils.refetch()` or a Query invalidation settles, an on-demand Query Collection behind a LEFT-joined, ordered, limited live query must show the window over the current backend rows. The design under test: query-db reports `invalidateLoadedSubsets()` when an ordered page changes keys and either has a later page or ends earlier in the order than before, and each ordered loader runs its bounded prefix repair. Its model is a pure top-k of marked backend rows. A fixed and a random campaign generate rounds of marker flips (which serve continuation pages), backend changes while the provider lags, a second live query (`observe`) whose own demand can load a page's new row first, optional Query invalidation with further changes, and a refetch, across one or two order terms, a numeric or locale-collated string order, limits 1–3, offsets 0–1, and a static or function Query key. With an unchanged backend, a repeated refetch must serve no more rows, start at most one first-page request, and report nothing. A refetch or invalidation must never start a new full-source request. A coverage law runs at the same checkpoints: every backend row from the start of the order through the window's last row (every row, when the window is short) must be in the source Collection with its current values, so a latent gap fails before a marker exposes it; a named replay of the issue trace with the moved row unmarked passes the window law and fails only the coverage law when Query DB never reports. The grammar can hold an optimistic update across a refetch, with the checkpoint moved to its release, and a premise-directed campaign under a hold (other rows observed, a row moved ahead of the page's end, a new row further ahead, a write to the last-sorting observed row) requires a report during a hold in every case and fails a skip that reads rows held back behind the transaction; the general campaigns did not. The general grammar holds optimistic deletes as well as updates. The generated campaigns declare reach (minimum 3) for a lagging refetch after a continuation, one during a hold, and a refetch that reported while the window was full, where the loader decides between ignoring the report and repairing. A named replay of a refetch during a held delete left the window short after release, also with this PR's report and overlap changes reverted, until the fix that resumes a continuation after a synchronous answer; no window is checked during a hold, where #2148 and #2169 remain open. Reports (`loadedSubsets:invalidated`) are bounded: a step reports at most once, and only when some ordered request was answered again with different rows; named replays require no report when a one-page window, or a window's last page, only loses a row, and a sync-boundary replay requires a report when a page ending on a row without an order value has its end move earlier (on the earlier gate, which read that page as empty, it reported nothing), and another requires a report when the provider returns a one-page window's rows in reverse order (the earlier gate took the last array row as the page's end). When a page's end moves earlier, Query DB reports. A loader whose window lies inside its own first page skips the repair only while its page read past the window or it read the tie of the current end; after an insert into a top-N list moves the end, a named replay requires the loader to prove the window again with bounded reads, no full-source read, and an unchanged Query count. A filtered one-page window whose page end moves past an unseen row must still repair (found by the random campaign; a loader that always skips fails it, the f9 tie replay, and the ordered-work oracle's signalling replays). The skip also refuses while the source holds accepted sync rows back: a replay refetches while an optimistic update persists, so the report arrives before the refetched rows are visible, and a skip that reads those older rows fails it. The ordered loader state oracle checks that a prefix read issued before an ordering invalidation, which settles after the bounded repair, does not retire the repair's prefix; the repair-first order failed before that fix. A `loadSubset` call that acquires a released, invalidated page can report, because Query then refetches that page on its own. A failed repair shows only the last window or the right one. Named replays cover the issue trace after a refetch and after an invalidation, with a page-capping provider, with another live query loading the page's new row first, with an empty later page, with a page whose end moves earlier while a tie request holds its old last row, a second refetch after an invalidation, a repair failure, and a provider that honors neither cursor nor offset. Never signalling, the old overlap-to-full-source rule, ignoring empty later pages, reporting on any page change, removing the repair loop guard, and reporting once per page each fail it by assertion. A replay makes a page's application fail and checks the window shows its last result or the right one, then the next refetch restores it; recording a page before its rows are accepted cannot be distinguished, because the failed attempt reports on its own. Open: the report laws bound reports but do not require each one; a page whose end moves earlier reports even with unique order values, where no row can be missing; it does not check whether a repair waits for in-flight work; the generated campaigns use an uncapped cursor provider only (an offset-only provider and a page-capping provider each hit a loader defect that reproduces on main: #2182, #2183); the generated campaigns do not check the window after a marker flip (a named replay checks the trace where the loader stopped refilling after a joined-side removal, which the repair-retirement fix below closed); a failed repair still recovers through the loader's full-source retry; interval, focus, and reconnect refetches share the invalidation path but are not driven; NaN order values have no executable witness; overlapping `refetch: true` acquisitions on one Query hash are counted, so the first to finish does not clear the loop guard for the other, but no witness separates that from a plain set; a Query refetch that joins a running `refetch: true` acquisition on the same hash does not report, and no witness reaches it (#2185); follower tabs of a multi-tab persisted collection never receive the report (#2174). The offset-only and page-capping loader defects have `it.fails` witnesses in the oracle. A finished bounded repair used to discard its boundary when its retirement made Query DB remove a row the loader had sent; the loader then restarted paging with a smaller first page that could not retire the repair's prefix, so every later refetch read both. A named replay found by the random campaign (at four times the run count) and an ordered loader state witness fail on that, and the fix also closed the joined-side refill defect. At four times the run count, the random campaign found a short window that stopped paging before the last row of the order: a report's repair released continuation Queries, and Query DB answered their reacquisition synchronously from its cache, so the graph turn inside each request asked for more while the loader was still requesting and nothing resumed the staged continuation. Only the coverage law failed. A named replay and an ordered loader state witness (a synchronous answer whose writes the graph already processed must continue the chain) fail on that, and pass with the fix. The loader side of the report also has a second owner: the ordered-work oracle's `refetch-signal` provider re-runs each served request alone and reports when any answer changed, across that oracle's generated plans, and a signalling refetch must start no full-source request unless a bounded page in that step ended on a null or NaN rank, which takes the documented full-source fallback (a random seed found that case). Never signalling and the old overlap-to-full-source rule each fail it. It excludes a peer live query, which can keep a short window after a refetch (#2155). The persisted-collection oracle checks that a source's report reaches the Collection through the managed cache-generation wrapper and that a retired source session's report is dropped. Remaining work for this class is tracked in #2186. | -| Ordered acquisition | [pagination](https://github.com/TanStack/db/blob/main/packages/db/tests/query/pagination-oracle.property.test.ts), [ordered work](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-work-oracle.property.test.ts), [ordered lifecycle](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-lifecycle-oracle.property.test.ts), [ordered loader state](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-source-loader-state-oracle.test.ts), [graph scheduler](https://github.com/TanStack/db/blob/main/packages/db/tests/query/scheduler.test.ts), [issue #1880 ordered-repair review](oracle-reviews/issue-1880-ordered-repair.md), [issue #1882 custom local collation review](oracle-reviews/issue-1882-custom-local-collation.md), [issue #1898 relation-filter review](oracle-reviews/issue-1898-relation-filter.md), [PR #1909 external review](oracle-reviews/pr-1909-external-review.md), [eager ordered-window repair review](oracle-reviews/perf-ordered-window-repair.md), [reversed index reads](https://github.com/TanStack/db/blob/main/packages/db/tests/reverse-index-oracle.property.test.ts) | Complete finite provider results, inherited locale and bounded/unbounded custom collation with exact request options, query-level inheritance across source aliases, real lexical/numeric disagreement, custom comparator ties, scan/index paths, pending windows, ties/nulls, lease ownership, Effect callback gates, including an indexed source update during Effect startup, and documented repair timing. Direct LEFT-joined filters cover local finite prefixes, pending child demand, child removal, and anti-join publication with custom full-source and unindexed fallback loading; unrelated include demand cannot stall the root continuation, and an empty joined replay wakes held Effect callbacks; remote relation hints remain outside this owner. Graph scheduler checks coalescing, dependency order, synchronous loader input, and repeated-alias failure ownership. The ordered-work oracle checks that a synchronous root refill reaches D2 before joined publication; a mutant that omitted the post-loader graph step failed at the second child demand. Custom collation proves local full-source acquisition only; it does not establish provider cursor capability or backend collation fidelity. Sibling-source input during a synchronous ordered continuation returns to graph work before another ordered acquisition. Request completion is not proof of unrequested source extent. A window move during existing joined demand waits for its root continuation, including an independently held root request; current demand failure rejects the move, while retirement and an obsolete rejection do not. A bounded ordered repair resumes its refill after joined demand settles. The ordered-source-loader state test checks that an explicit failed-request retry blocked by joined demand retains its generation until it can issue a request. Multiple simultaneously pending joined plans during a window move and a public-query failed-retry overlap still need dedicated witnesses. The ordered-work oracle also checks an eager indexed window over generated upserts and deletes: the rows equal the filtered, sorted model, and each change delivers a bounded number of source rows; an unbounded or always-full repair read fails it. Its filter dimension adds a function filter and a LEFT-joined filter, and a campaign fixed to the joined premise checks that a window refilled past a distant row the query already received holds the first eligible rows. Joined filters also run as anti-joins, change eligibility through the marker or the root row's join key, and drive an Effect beside the Collection. A full-join filter keeps every root row and adds a null-ranked rootless row for each marker no root row matches; on main a bounded root prefix showed markers whose root rows were not yet loaded as unmatched rows, and restoring the old inner-or-right full-source rule fails eight tests. Null placement is a generated dimension, with the index built to match, and a query that ignores it fails six tests. A full-join campaign has few or no filler rows, a matched null-ranked root row and at least one orphan marker in every case, so its windows reach the null group where rootless rows and null-ranked root rows tie; a reconstruction check samples exactly the cases its fixed-seed run executes and requires such windows (19 rootless and 7 mixed null-group windows of 40 at one times the run count), and the Effect runs beside the Collection. Full joins read their full source, so only the rows law covers them; paging them under the old join rule fails the window law. Pinned cases cover a null-ranked root row past a prefix that ends in the null group. Paging a nulls-last full join passed this oracle but was not adopted: it cost more per delete when the window reached the null group, which no work law here measures, and #2130 tracks full-join paging. Joined filters can match a row through several markers, and the model keeps one result row per matched pair; a model that collapses them fails. The eager harness also runs an on-demand source whose provider follows the LoadSubsetOptions contract (offset only without a cursor, the applied receipt returned). Its test provider combines a sync adapter and its provider (glossary model alignment rule 2); a held request is a physical acquisition whose provider answer is delayed, a served range is the prefix and boundary one answer establishes (a model abstraction), and the peer is a second live-query Collection. The provider sorts by the requested order and keeps its own ledger of sent rows. A forwarding provider sends live changes only for rows it sent or rows that match a served subset, and its served rows obey the work bound; a loader that asks for 100 extra rows fails it. A refetching provider, like query-db-collection, sends nothing until a refetch. The refetch keeps every served ordered range whole, as the LoadSubset contract requires: it sends every row from the start of the order through the last order value a limited page returned, and rows in no range leave; the rows law over the model applies once no change is pending. After every step on either provider, the Collection must hold exactly the rows in the provider's ledger, and the query and the Effect must show their window over those rows; a mutant that drops marker removals fails this law at steps 0 and 1 of the fixed refetching campaigns, earlier than the model's rows law, and in one more campaign. A holding provider answers each load only at an `answer` step, from the rows at that moment, so live changes arrive while loads are in flight. While a load is in flight, the query and its peer either keep the window they showed at the previous checkpoint or show distinct ledger rows that pass the query, in its order; the Effect is not checked then. That is weaker than the architecture's barrier rule, under which a repair or window move keeps the last public snapshot until it settles: a barrier law fails today on on-demand joined plans, where a visible-delete repair and a held `setWindow` publish before they settle (#2131; the law and traces are kept for that fix). The provider records a request as served only once it answers it, and an abort (or an adapter unload) of a held request follows `LoadSubsetOptions.signal`: a cooperative provider installs none of its rows and its load fulfills at once, while a provider that cannot cancel in-flight work (`late` cancellation, generated) answers it anyway and its load fulfills after those rows become visible. A `release-peer` step cleans up the peer while its loads are held, which aborts them; each request counts once although its acquisition release both aborts and unloads it. The fixed held campaign reaches 5 aborted held requests and 1 late answer, and its reconstruction check requires at least 3 cooperative cancellations and 1 late answer. A held load rejects when its applied receipt rejects, and the step then fails, because no signal or cleanup rejects a receipt in these histories; that branch is unwitnessed. The held campaign pins a forwarding provider, so `release-peer` never reaches a refetching provider; that branch is unwitnessed. A request whose signal is already aborted when the provider receives it is never reached; that branch is unwitnessed. The held campaign uses a forwarding provider and an indexed source filter or a joined filter, deletes three visible rows and then upserts an eligible row with an id no other command uses; its reconstruction check runs in the same test as the fixed-seed campaign and reads its per-case counts: 35 of 40 cases issue a held request, and every one of them changes rows while it is in flight. Loading must reach a fixed point: past a request budget the provider rejects and records it, and the initial load, every step and the end of each history assert the record is clear (a loader that re-requests a prefix it already holds can loop without reaching the provider, which this cannot see). A bounded step makes at most four provider requests, and each consumer's initial proven-prefix load is bounded separately, in total transfers (a repeated request for the same prefix counts) and in distinct rows (at most the source's size); a loader that asks for 100 extra rows on every page fails that bound and the initial-load work law. Loader mutants that never resume after a boundary request, or that ignore a load request while another load is pending, fail it; a mutant that drops a window move blocked behind a pending load still survives, even with the settlement check on held moves; whether it is equivalent is open. Open (#2148): on an on-demand source with a joined or anti-joined filter, an optimistic delete of the visible row leaves the window short until the delete persists, because the next row is never loaded. On an on-demand source with a source filter, and on an eager source with any filter, the same delete refills the window correctly; these are the contrast cases. Owner: this oracle. Needed witness: the optimistic-delete dimension (kept as a patch for #2148) in the grammar, plus a pinned replay of the anti-join trace. A refetch that re-runs each page alone, as query-db-collection does today, leaves a row that moved between two pages unloaded; random seeds found that trace, a named replay fails on it, and a query-db receiving witness reproduces it (#2136 tracks the provider fix). An Electric-style provider, which skips updates to rows it never sent, is not modeled; a mocked-stream witness shows a row moving into a served ordered range never arriving under the default replica mode (#2138). At one times the run count, the campaigns reach 87 refetch steps: 39 change the window, 6 remove a visible row, and 13 lead to a further provider request. A random seed found a refetch that empties a joined window, after which the loader refills it over several page requests and publishes `[]`, one row, then the final rows; a named replay keeps that trace for #2131. No incorrect final window has appeared on that path, and no loader mutant tried so far is caught only by it; the work law does not cover it. For an on-demand joined or anti-joined filter, the initial load must not request the whole source when the first window's worth of source rows in query order all pass the join and no boundary value is null or NaN; 35 executed cases meet that premise, and a compiler that reads the whole source for every joined filter fails eight tests on this law alone. The rows served by that initial load stay within the prefix, its boundary ties and a refill, for each consumer. Later steps of those plans have no work bound. An unindexed join filter over it left the window empty on main, until the plan counted the rows it still needs; restoring the old guard fails the campaigns and a pinned replay. A step that changes the visible rows must publish, and its subscriber starts with every visible key, so a removal-only change reaches it; mutants that drop update-only or removal-only publications fail. A step may publish one intermediate window per provider request it made, and only its last publication must show the final window, because a repair or refill can publish after each round trip (#2131); the content of those intermediate windows is unchecked. The model keeps its own copy of the markers, and full-join histories can insert the root rows of orphan markers. A premise-directed campaign inserts a fresh row and then changes an initial row's eligibility alone, so the row leaves the window only through the join; it and a named replay fail when a window row past the loader's settled boundary counts toward a full window. One case in four adds a second live query over the same sources with a larger window, and its rows law holds at every checkpoint; the work laws do not apply to those cases. #2155: a second live-query Collection over a refetching on-demand provider kept a short joined window after a refetch removed a tied row (the peer showed ranks `[0, 0]` where the model shows `[0, 0, 100]`). The loader fix that resumes a staged continuation after a synchronous answer the graph already processed closed that trace. The pinned replay of that indexed trace now expects the right window. Still open: an unindexed peer over a refetching provider keeps a short joined window (the peer shows `[100]` where the model shows `[100, 101]`), which also fails without that fix; each campaign therefore still excludes the peer and refetching pairing, as its last map. Owner: this oracle. Needed witness: a pinned replay of the unindexed trace flipped to pass, and the pairing restored to the grammar. Limits: Effects are checked for rows after each step, not for publication counts or positions. Histories move the first query's window with `setWindow` (150 moves at one times the run count, 56 on demand), and its rows law and ledger law then follow the moved window; the Effect and the peer keep theirs. On a holding provider every window-move promise must have fulfilled after the final drain. A refetching provider models query-db's unload: it drops the unloaded request and removes rows that no remaining request's last answer holds. The per-step work law applies only to forwarding, immediate, single-consumer cases: it skips refetching, holding and peer cases and move steps; a two-term indexed move scanned the source four times, which no law here bounds. Multi-term joined plans stay outside this harness, on both paths; on the on-demand path, the provider's transfer is bounded only for indexed single-term source-predicate plans. Joined windows compare order values, so a wrong row tied with the expected one passes; a window row tied with the boundary counts as proven before its tie request settles, which no witness exercises. A boundary that cannot advance past a window row would repeat one page per new key; reading the boundary's range locally should always reach that row, but no witness shows it. Joined ties are compared as any valid first rows; whether source-filter ties may be fixed to the source key is open. The same grammar includes NaN and null ranks and a second-order term, and bounds both rows read and rows delivered, including the initial load and index refills; a scan law limits each change to one pass over the source. A pinned case checks that an indexed descending window places nulls first, and a lifecycle witness checks that an eager full-source window is ready with its rows at creation when its source answers synchronously. The initial fallback cells check indexed and unindexed locale strings with distinct and tied values, plus a null boundary after a warm ordered request. The warm null-boundary case crosses preload-started and constructor-started live-query sync runs: the former completes at the synchronous cut, while the latter resumes deferred on-demand acquisition asynchronously and publishes after settlement. They require an unfiltered full-source request, no public row at fallback entry, and the expected row and status at the immediate readiness check. A Promise fallback stays loading at that checkpoint. A React receiving witness checks an eager locale-string `findOne()` at the first layout commit. A Query DB receiving witness checks warm cached subsets: both raw adapter calls return literal `true` with the source rows visible at each return, and the live query is immediately ready without a fetch. The same fallback cells observe subscriber changes from before the first source write: literal-true and Promise fallbacks each publish one nonempty final-window insert after the fallback settles, and the callback sees the complete row. Promise cells show no nonempty event at the immediate cut; a duplicate-event fault fails all eight cells. Other adapters and a real on-demand React first commit remain unwitnessed. Full-source work after initial publication and repair remain outside this synchronous guarantee. A separate post-publication oracle crosses marker deletion and retargeting with literal, Promise, and held-applied-receipt fallback responses. It also settles the child demand before holding the receipt, so the public live-query loading flag isolates the root fallback. The model checks final rows and net events, while event replay checks every callback snapshot; a retained or empty intermediate window is allowed. A loader fault that ignores the pending receipt fails at the held loading-flag check. Two literal-true cleanup-failure histories require that the replacement still publishes and the adapter error reaches the public query status without making the unrelated marker commit throw; the provider's applied receipts are recorded and compared outside provider callbacks. The Promise case checks eventual coherence on an asynchronous callback path, without claiming a separate timing cut. Four held-Promise histories apply fallback writes, change another join marker before provider settlement, and compare each callback and the final winner; an injected stale extra publication fails the event comparison. Other marker actions and overlapping root-source changes remain open; the provider has no later write or rejection, so this schedule does not establish when the loader claims full-source authority. The reversed-index oracle checks every reversed read's order and filter work for both null placements. | +| Ordered acquisition | [pagination](https://github.com/TanStack/db/blob/main/packages/db/tests/query/pagination-oracle.property.test.ts), [ordered work](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-work-oracle.property.test.ts), [ordered lifecycle](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-lifecycle-oracle.property.test.ts), [ordered loader state](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-source-loader-state-oracle.test.ts), [graph scheduler](https://github.com/TanStack/db/blob/main/packages/db/tests/query/scheduler.test.ts), [issue #1880 ordered-repair review](oracle-reviews/issue-1880-ordered-repair.md), [issue #1882 custom local collation review](oracle-reviews/issue-1882-custom-local-collation.md), [issue #1898 relation-filter review](oracle-reviews/issue-1898-relation-filter.md), [PR #1909 external review](oracle-reviews/pr-1909-external-review.md), [eager ordered-window repair review](oracle-reviews/perf-ordered-window-repair.md), [reversed index reads](https://github.com/TanStack/db/blob/main/packages/db/tests/reverse-index-oracle.property.test.ts) | Complete finite provider results, inherited locale and bounded/unbounded custom collation with exact request options, query-level inheritance across source aliases, real lexical/numeric disagreement, custom comparator ties, scan/index paths, pending windows, ties/nulls, lease ownership, Effect callback gates, including an indexed source update during Effect startup, and documented repair timing. Direct LEFT-joined filters cover local finite prefixes, pending child demand, child removal, and anti-join publication with custom full-source and unindexed fallback loading; unrelated include demand cannot stall the root continuation, and an empty joined replay wakes held Effect callbacks; remote relation hints remain outside this owner. Graph scheduler checks coalescing, dependency order, synchronous loader input, and repeated-alias failure ownership. The ordered-work oracle checks that a synchronous root refill reaches D2 before joined publication; a mutant that omitted the post-loader graph step failed at the second child demand. Custom collation proves local full-source acquisition only; it does not establish provider cursor capability or backend collation fidelity. Sibling-source input during a synchronous ordered continuation returns to graph work before another ordered acquisition. Request completion is not proof of unrequested source extent. A window move during existing joined demand waits for its root continuation, including an independently held root request; current demand failure rejects the move, while retirement and an obsolete rejection do not. A bounded ordered repair resumes its refill after joined demand settles. The ordered-source-loader state test checks that an explicit failed-request retry blocked by joined demand retains its generation until it can issue a request. Multiple simultaneously pending joined plans during a window move and a public-query failed-retry overlap still need dedicated witnesses. The ordered-work oracle also checks an eager indexed window over generated upserts and deletes: the rows equal the filtered, sorted model, and each change delivers a bounded number of source rows; an unbounded or always-full repair read fails it. Its filter dimension adds a function filter and a LEFT-joined filter, and a campaign fixed to the joined premise checks that a window refilled past a distant row the query already received holds the first eligible rows. Joined filters also run as anti-joins, change eligibility through the marker or the root row's join key, and drive an Effect beside the Collection. A full-join filter keeps every root row and adds a null-ranked rootless row for each marker no root row matches; on main a bounded root prefix showed markers whose root rows were not yet loaded as unmatched rows, and restoring the old inner-or-right full-source rule fails eight tests. Null placement is a generated dimension, with the index built to match, and a query that ignores it fails six tests. A full-join campaign has few or no filler rows, a matched null-ranked root row and at least one orphan marker in every case, so its windows reach the null group where rootless rows and null-ranked root rows tie; a reconstruction check samples exactly the cases its fixed-seed run executes and requires such windows (19 rootless and 7 mixed null-group windows of 40 at one times the run count), and the Effect runs beside the Collection. Full joins read their full source, so only the rows law covers them; paging them under the old join rule fails the window law. Pinned cases cover a null-ranked root row past a prefix that ends in the null group. Paging a nulls-last full join passed this oracle but was not adopted: it cost more per delete when the window reached the null group, which no work law here measures, and #2130 tracks full-join paging. Joined filters can match a row through several markers, and the model keeps one result row per matched pair; a model that collapses them fails. The eager harness also runs an on-demand source whose provider follows the LoadSubsetOptions contract (offset only without a cursor, the applied receipt returned). Its test provider combines a sync adapter and its provider (glossary model alignment rule 2); a held request is a physical acquisition whose provider answer is delayed, a served range is the prefix and boundary one answer establishes (a model abstraction), and the peer is a second live-query Collection. The provider sorts by the requested order and keeps its own ledger of sent rows. A forwarding provider sends live changes only for rows it sent or rows that match a served subset, and its served rows obey the work bound; a loader that asks for 100 extra rows fails it. A refetching provider, like query-db-collection, sends nothing until a refetch. The refetch keeps every served ordered range whole, as the LoadSubset contract requires: it sends every row from the start of the order through the last order value a limited page returned, and rows in no range leave; the rows law over the model applies once no change is pending. After every step on either provider, the Collection must hold exactly the rows in the provider's ledger, and the query and the Effect must show their window over those rows; a mutant that drops marker removals fails this law at steps 0 and 1 of the fixed refetching campaigns, earlier than the model's rows law, and in one more campaign. A holding provider answers each load only at an `answer` step, from the rows at that moment, so live changes arrive while loads are in flight. While a load is in flight, the query and its peer either keep the window they showed at the previous checkpoint or show distinct ledger rows that pass the query, in its order; the Effect is not checked then. That is weaker than the architecture's barrier rule, under which a repair or window move keeps the last public snapshot until it settles: a barrier law fails today on on-demand joined plans, where a visible-delete repair and a held `setWindow` publish before they settle (#2131; the law and traces are kept for that fix). The provider records a request as served only once it answers it, and an abort (or an adapter unload) of a held request follows `LoadSubsetOptions.signal`: a cooperative provider installs none of its rows and its load fulfills at once, while a provider that cannot cancel in-flight work (`late` cancellation, generated) answers it anyway and its load fulfills after those rows become visible. A `release-peer` step cleans up the peer while its loads are held, which aborts them; each request counts once although its acquisition release both aborts and unloads it. The fixed held campaign reaches 5 aborted held requests and 1 late answer, and its reconstruction check requires at least 3 cooperative cancellations and 1 late answer. A held load rejects when its applied receipt rejects, and the step then fails, because no signal or cleanup rejects a receipt in these histories; that branch is unwitnessed. The held campaign pins a forwarding provider, so `release-peer` never reaches a refetching provider; that branch is unwitnessed. A request whose signal is already aborted when the provider receives it is never reached; that branch is unwitnessed. The held campaign uses a forwarding provider and an indexed source filter or a joined filter, deletes three visible rows and then upserts an eligible row with an id no other command uses; its reconstruction check runs in the same test as the fixed-seed campaign and reads its per-case counts: 35 of 40 cases issue a held request, and every one of them changes rows while it is in flight. Loading must reach a fixed point: past a request budget the provider rejects and records it, and the initial load, every step and the end of each history assert the record is clear (a loader that re-requests a prefix it already holds can loop without reaching the provider, which this cannot see). A bounded step makes at most four provider requests, and each consumer's initial proven-prefix load is bounded separately, in total transfers (a repeated request for the same prefix counts) and in distinct rows (at most the source's size); a loader that asks for 100 extra rows on every page fails that bound and the initial-load work law. Loader mutants that never resume after a boundary request, or that ignore a load request while another load is pending, fail it; a mutant that drops a window move blocked behind a pending load still survives, even with the settlement check on held moves; whether it is equivalent is open. Open (#2148): on an on-demand source with a joined or anti-joined filter, an optimistic delete of the visible row leaves the window short until the delete persists, because the next row is never loaded. On an on-demand source with a source filter, and on an eager source with any filter, the same delete refills the window correctly; these are the contrast cases. Owner: this oracle. Needed witness: the optimistic-delete dimension (kept as a patch for #2148) in the grammar, plus a pinned replay of the anti-join trace. A refetch that re-runs each page alone, as query-db-collection does today, leaves a row that moved between two pages unloaded; random seeds found that trace, a named replay fails on it, and a query-db receiving witness reproduces it (#2136 tracks the provider fix). An Electric-style provider, which skips updates to rows it never sent, is not modeled; a mocked-stream witness shows a row moving into a served ordered range never arriving under the default replica mode (#2138). At one times the run count, the campaigns reach 87 refetch steps: 39 change the window, 6 remove a visible row, and 13 lead to a further provider request. A random seed found a refetch that empties a joined window, after which the loader refills it over several page requests and publishes `[]`, one row, then the final rows; a named replay keeps that trace for #2131. No incorrect final window has appeared on that path, and no loader mutant tried so far is caught only by it; the work law does not cover it. For an on-demand joined or anti-joined filter, the initial load must not request the whole source when the first window's worth of source rows in query order all pass the join and no boundary value is null or NaN; 35 executed cases meet that premise, and a compiler that reads the whole source for every joined filter fails eight tests on this law alone. The rows served by that initial load stay within the prefix, its boundary ties and a refill, for each consumer. Later steps of those plans have no work bound. An unindexed join filter over it left the window empty on main, until the plan counted the rows it still needs; restoring the old guard fails the campaigns and a pinned replay. A step that changes the visible rows must publish, and its subscriber starts with every visible key, so a removal-only change reaches it; mutants that drop update-only or removal-only publications fail. A step may publish one intermediate window per provider request it made, and only its last publication must show the final window, because a repair or refill can publish after each round trip (#2131); the content of those intermediate windows is unchecked. The model keeps its own copy of the markers, and full-join histories can insert the root rows of orphan markers. A premise-directed campaign inserts a fresh row and then changes an initial row's eligibility alone, so the row leaves the window only through the join; it and a named replay fail when a window row past the loader's settled boundary counts toward a full window. One case in four adds a second live query over the same sources with a larger window, and its rows law holds at every checkpoint; the work laws do not apply to those cases. #2155: a second live-query Collection over a refetching on-demand provider kept a short joined window after a refetch removed a tied row (the peer showed ranks `[0, 0]` where the model shows `[0, 0, 100]`). The loader fix that resumes a staged continuation after a synchronous answer the graph already processed closed that trace. The pinned replay of that indexed trace now expects the right window. Still open: an unindexed peer over a refetching provider keeps a short joined window (the peer shows `[100]` where the model shows `[100, 101]`), which also fails without that fix; each campaign therefore still excludes the peer and refetching pairing, as its last map. Owner: this oracle. Needed witness: a pinned replay of the unindexed trace flipped to pass, and the pairing restored to the grammar. Limits: Effects are checked for rows after each step, not for publication counts or positions. Histories move the first query's window with `setWindow` (150 moves at one times the run count, 56 on demand), and its rows law and ledger law then follow the moved window; the Effect and the peer keep theirs. On a holding provider every window-move promise must have fulfilled after the final drain. A refetching provider models query-db's unload: it drops the unloaded request and removes rows that no remaining request's last answer holds. The per-step work law applies only to forwarding, immediate, single-consumer cases: it skips refetching, holding and peer cases and move steps; a two-term indexed move scanned the source four times, which no law here bounds. Multi-term joined plans stay outside this harness, on both paths; on the on-demand path, the provider's transfer is bounded only for indexed single-term source-predicate plans. Joined windows compare order values, so a wrong row tied with the expected one passes; a window row tied with the boundary counts as proven before its tie request settles, which no witness exercises. A boundary that cannot advance past a window row would repeat one page per new key; reading the boundary's range locally should always reach that row, but no witness shows it. Joined ties are compared as any valid first rows; whether source-filter ties may be fixed to the source key is open. The same grammar includes NaN and null ranks and a second-order term, and bounds both rows read and rows delivered, including the initial load and index refills; a scan law limits each change to one pass over the source. A pinned case checks that an indexed descending window places nulls first, and a lifecycle witness checks that an eager full-source window is ready with its rows at creation when its source answers synchronously. The initial fallback cells check indexed and unindexed locale strings with distinct and tied values, plus a null boundary after a warm ordered request. The warm null-boundary case crosses preload-started and constructor-started live-query sync runs: the former completes at the synchronous cut, while the latter resumes deferred on-demand acquisition asynchronously and publishes after settlement. They require an unfiltered full-source request, no public row at fallback entry, and the expected row and status at the immediate readiness check. A Promise fallback stays loading at that checkpoint. A React receiving witness checks an eager locale-string `findOne()` at the first layout commit. A Query DB receiving witness checks warm cached subsets: both raw adapter calls return literal `true` with the source rows visible at each return, and the live query is immediately ready without a fetch. The same fallback cells observe subscriber changes from before the first source write: literal-true and Promise fallbacks each publish one nonempty final-window insert after the fallback settles, and the callback sees the complete row. Promise cells show no nonempty event at the immediate cut; a duplicate-event fault fails all eight cells. Other adapters and a real on-demand React first commit remain unwitnessed. Full-source work after initial publication and repair remain outside this synchronous guarantee. A separate post-publication oracle crosses marker deletion and retargeting with literal, Promise, and held-applied-receipt fallback responses. It also settles the child demand before holding the receipt, so the public live-query loading flag isolates the root fallback. The model checks final rows and net events, while event replay checks every callback snapshot; a retained or empty intermediate window is allowed. A loader fault that ignores the pending receipt fails at the held loading-flag check. Two literal-true cleanup-failure histories require that the replacement still publishes and the adapter error reaches the public query status without making the unrelated marker commit throw; the provider's applied receipts are recorded and compared outside provider callbacks. The Promise case checks eventual coherence on an asynchronous callback path, without claiming a separate timing cut. Four held-Promise histories apply fallback writes, change another join marker before provider settlement, and compare each callback and the final winner; an injected stale extra publication fails the event comparison. Other marker actions and overlapping root-source changes remain open; the provider has no later write or rejection, so this schedule does not establish when the loader claims full-source authority. The reversed-index oracle checks every reversed read's order and filter work for both null placements. The retirement-invalidation lane (`ordered-loader.retirement-invalidation`) varies index availability, limit 1/2, synchronous repair, joined deferral, and needed rows. Its external request ledger requires a newer authoritative start when adapter release delivers a distinct ordering invalidation during successful bounded-repair retirement. Fixed/random campaigns each reach 200 callback histories. A receiving driver (`ordered-loader.retirement-receiver`) in the same owner creates a real on-demand Collection and live-query Collection, varies indexed/unindexed, limit 1/2 and literal/Promise provider answers, and moves the repaired prefix behind a still-uninstalled provider row from `unloadSubset`. It applies the same progress judgment, checks public rows against a provider full sort, and requires one complete replacement publication; fixed/random campaigns each reach 200 real callbacks. The original loader loses the newer debt and shows id 1/rank 10 instead of id 3/rank 3 in the shrunk width-one receiver. The final transition preserves debt when the repair epoch moved during retirement. Original-defect calibration rejects both drivers. A smaller debt-preservation candidate passed the publication judgment, so immediate recursive dispatch was not retained. The original core/real progress and public-row RED preceded production edits; the stronger publication recorder was calibrated against baseline after the first candidate. The replay-continuation lane (`ordered-loader.replay-continuation`) preserves deferred refill provenance across replay bookkeeping: bookkeeping does not consume a graph continuation; truncate clears it and only current-generation requests can stage it. Directed handoff/reset campaigns and the exact #2296 replay reject the original bookkeeping ablation. This is a model/harness repair, not evidence of a production resource leak. Limits: one new qualifying invalidation per tested retirement callback; the real receiver has distinct finite numeric ranks, ascending root order, one source, one consumer, and no failure/truncate/disposal in that callback. Existing generated authority and replay owners retain their failure, reset/disposal, covering lease, no-retirement and peer neighbors. No claim about #2223 obsolete later cursor-demand retirement, arbitrary callback composition, or provider-specific network transports follows. The controlled and receiving grammars have distinct registered replay properties, so each seed and shrink path selects only its own driver. The no-leak calibration supplies a retained-lease observation to the same request-ledger comparison after an unmodified run of the same covering-repair history; it reads no private member names. | | Indexed predicate filtering | `packages/db/tests/query/index-path-collision-oracle.test.ts` | A bounded cross product of numeric values checks exact public keys before and after adding a nested-field BTree index. Direct predicates vary argument order, bound inclusivity, reversed operands, and bound field. Public Collection subscription and live-query callbacks run with and without the index. Selected-field ordering compares a direct JavaScript sort with a public `$selected` callback; restoring the dotted-key proxy cache made that comparison fail. The original compound grouping failed four indexed cases while scan controls passed. The original callback proxy caches also failed both unindexed public routes. A hostile cleanup control preserves the primary mismatch and secondary release error. Arbitrary path segments, nullish values, custom collation, and incremental publication need separate witnesses if claimed. | | Lazy target path identity | `packages/db/tests/query/compiler/lazy-targets.test.ts` | A focused same-source `UnionFrom`/`coalesce` witness requires both [`a.b`] and [`a`, `b`] demand targets. Restoring dotted-string deduplication drops the second target at the compiler boundary. A public on-demand adapter and row-publication history still need a separate witness. | diff --git a/packages/db/src/query/live/ordered-source-loader.ts b/packages/db/src/query/live/ordered-source-loader.ts index baa705d94b..4fecc5ee94 100644 --- a/packages/db/src/query/live/ordered-source-loader.ts +++ b/packages/db/src/query/live/ordered-source-loader.ts @@ -762,7 +762,13 @@ export class OrderedSourceLoader { if (!this.active || !hasSupersededAcquisition) { this.cancelRepairRetry() this.repairRetries = 0 - this.moveWork({ kind: `idle` }, [`repair`], `none`) + // Retirement can deliver a newer invalidation. This chain pays only + // its own epoch; the next pass must still repair that newer order. + this.moveWork( + { kind: `idle` }, + [`repair`], + this.repairNeedsReplacement() ? this.debt : `none`, + ) } } } diff --git a/packages/db/tests/oracle-config.ts b/packages/db/tests/oracle-config.ts index ca9c1a8927..cc0759b88d 100644 --- a/packages/db/tests/oracle-config.ts +++ b/packages/db/tests/oracle-config.ts @@ -53,6 +53,9 @@ const staticOracleProperties = [ `query-db.refetch-ordered-prefix`, `ordered-loader.request-authority`, `ordered-loader.unproven-report`, + `ordered-loader.replay-continuation`, + `ordered-loader.retirement-invalidation`, + `ordered-loader.retirement-receiver`, `replay.ordered-window-faults`, `replay.ordered-failed-replay`, `replay.ordered-preload-recovery`, diff --git a/packages/db/tests/oracle-replay-manifest.ts b/packages/db/tests/oracle-replay-manifest.ts index 5de3ae76b1..d8c5af00f0 100644 --- a/packages/db/tests/oracle-replay-manifest.ts +++ b/packages/db/tests/oracle-replay-manifest.ts @@ -347,7 +347,7 @@ const ownerGroups: ReadonlyArray< [ `db/tests/query/ordered-source-loader-state-oracle.test.ts`, `ordered-loader`, - `request-authority unproven-report`, + `request-authority unproven-report replay-continuation retirement-invalidation retirement-receiver`, ], [ `db-sqlite-persistence-core/tests/sqlite-core-adapter-cli-runtime.test.ts`, diff --git a/packages/db/tests/query/ordered-source-loader-state-oracle.test.ts b/packages/db/tests/query/ordered-source-loader-state-oracle.test.ts index df1ceebb1e..6952cf418b 100644 --- a/packages/db/tests/query/ordered-source-loader-state-oracle.test.ts +++ b/packages/db/tests/query/ordered-source-loader-state-oracle.test.ts @@ -6,19 +6,23 @@ import { } from '../../src/query/live/ordered-source-loader.js' import { PropRef } from '../../src/query/ir.js' import { createOracleReach } from '../oracle-reach.js' -import { captureMicrotaskErrors } from '../utils.js' +import { captureMicrotaskErrors, flushPromises } from '../utils.js' import { oraclePropertyOptions, oracleRuns, readOracleRunConfig, } from '../oracle-config.js' import { declareOracle } from '../oracle-declaration.js' +import { createCollection } from '../../src/collection/index.js' +import { createLiveQueryCollection } from '../../src/query/index.js' +import { BTreeIndex } from '../../src/indexes/btree-index.js' +import { evaluateReferenceExpression } from '../reference-expression-oracle.js' import { createDeferred, createOrderByInfo, } from './ordered-source-loader-support.js' import type { CollectionSubscription } from '../../src/collection/subscription.js' -import type { LoadSubsetOptions } from '../../src/types.js' +import type { LoadSubsetOptions, SyncConfig } from '../../src/types.js' import type { Observed, RequestOptions, @@ -62,8 +66,8 @@ function tieValue(options: RequestOptions): unknown { * - **No stale step:** a repair chain whose ordering epoch moved is replaced * before it issues another tie or refill. * - **Error identity:** a request that fails reports its own error through - * its participant, or a release's cleanup error. A read the loader starts - * while handling it cannot replace that error. A dead request's failure + * its participant. Neither a release's cleanup error nor a read the loader + * starts while handling it can replace that error. A dead request's failure * belongs to its replay, and one a replay reacquired is absorbed. * - **Cost:** once loading is quiet and a bounded repair is available, an * invalidation reads no full source, and it issues at most one chain (in @@ -320,6 +324,9 @@ type AuthorityCase = { releaseReenters: boolean // The first release callback disposes the loader, as adapter cleanup can. releaseDisposes: boolean + // A source-order invalidation reenters the first retirement callback. Unlike a + // delete caused by retirement itself, it owes a new ordering repair. + releaseInvalidates?: boolean history: Array } @@ -495,10 +502,207 @@ type AuthorityObservation = { costChecks: number hungRequests: number offsetWindows: number + releaseInvalidations: number + replayContinuationHandoffs: number + // Current failed requests whose participant reason reached the identity cut. + requestErrorChecks: number +} + +// Retirement cannot pay an invalidation that happened after the covering read. +// Both drivers supply only observed source events and subsequent request starts. +// The comparison does not inspect the loader's work or debt representation. +function checkRetirementProgress( + invalidations: number, + replaced: boolean, +): void { + if (invalidations > 0 && !replaced) { + throw new Error( + `retirement invalidation: ${invalidations} adapter invalidation reached the release pass, but no new authoritative read followed it`, + ) + } +} + +type RetirementReceiverCase = { + indexed: boolean + limit: 1 | 2 + synchronous: boolean +} + +// This receiving driver supplies the controlled ledger's premise through the +// actual Collection subscription: adapter unload changes a contributed row's +// leading order value. A finite provider keeps a later row uninstalled. Moving +// the current prefix behind it therefore requires a fresh authoritative read. +// The shared progress judgment runs after requests and graph work drain. Public +// rows are a second formulation: full provider sorting, without loader state. +async function checkRetirementReceiver( + scenario: RetirementReceiverCase, + reached: () => void, +): Promise { + type Row = { id: number; rank: number } + const truth = new Map( + [1, 2, 3, 4, 5].map((id) => [id, { id, rank: id }]), + ) + const installed = new Map() + const calls: Array = [] + let invalidations = 0 + let invalidatedAt = -1 + let armed = false + let sync!: Parameters[`sync`]>[0] + const source = createCollection({ + getKey: (row) => row.id, + syncMode: `on-demand`, + autoIndex: scenario.indexed ? `eager` : `off`, + defaultIndexType: BTreeIndex, + sync: { + sync: (operations) => { + sync = operations + operations.markReady() + return { + loadSubset: (options) => { + calls.push(options) + const deliver = () => { + if (options.signal?.aborted) return true + const matching = [...truth.values()] + .filter( + (row) => + (!options.where || + evaluateReferenceExpression(options.where, row) === + true) && + (!options.cursor || + evaluateReferenceExpression( + options.cursor.whereFrom, + row, + ) === true), + ) + .sort((left, right) => left.rank - right.rank) + const offset = options.cursor ? 0 : (options.offset ?? 0) + const rows = matching.slice( + offset, + options.limit === undefined + ? undefined + : offset + options.limit, + ) + operations.begin() + for (const value of rows) { + if (installed.get(value.id) === value) continue + operations.write({ + type: installed.has(value.id) ? `update` : `insert`, + value, + }) + installed.set(value.id, value) + } + return operations.commit() + } + return scenario.synchronous + ? deliver() + : Promise.resolve().then(async () => { + await deliver() + }) + }, + unloadSubset: () => { + if (!armed || invalidations > 0) return + invalidations++ + invalidatedAt = calls.length + reached() + // The first repair has read the next prefix. Move every row of + // that prefix after the provider's still-uninstalled next row. + operations.begin() + for (let id = 2; id <= scenario.limit + 1; id++) { + const value = { id, rank: 20 + id } + truth.set(id, value) + installed.set(id, value) + operations.write({ type: `update`, value }) + } + operations.commit() + }, + } + }, + }, + }) + const live = createLiveQueryCollection((q) => + q + .from({ row: source }) + .orderBy(({ row }) => row.rank) + .limit(scenario.limit) + .select(({ row }) => ({ id: row.id, rank: row.rank })), + ) + const publications: Array> = [] + const listener = live.subscribeChanges( + () => { + if (armed) + publications.push(live.toArray.map(({ id, rank }) => ({ id, rank }))) + }, + { includeInitialState: false }, + ) + try { + await live.preload() + armed = true + const moved = { id: 1, rank: 10 } + truth.set(1, moved) + installed.set(1, moved) + sync.begin() + sync.write({ type: `update`, value: moved }) + await sync.commit() + await flushPromises() + expect(invalidations, `real subscription retirement was reached`).toBe(1) + const failures: Array = [] + try { + checkRetirementProgress( + invalidations, + calls + .slice(invalidatedAt) + .some( + (options) => + options.refetch === true && + options.where === undefined && + options.cursor === undefined, + ), + ) + } catch (error) { + failures.push(error) + } + const expected = [...truth.values()] + .sort((left, right) => left.rank - right.rank) + .slice(0, scenario.limit) + try { + expect( + live.toArray.map(({ id, rank }) => ({ id, rank })), + `settled public window after retirement invalidation`, + ).toEqual(expected) + } catch (error) { + failures.push(error) + } + try { + // Both invalidations belong to one unfinished replacement. Publication + // cannot expose the first chain's rows after its retirement moved them. + expect(publications, `one complete replacement publication`).toEqual([ + expected, + ]) + } catch (error) { + failures.push(error) + } + if (failures.length > 0) { + throw new AggregateError( + failures, + `retirement receiving comparisons: ${failures.map((error) => (error instanceof Error ? error.message : String(error))).join(`; `)}`, + { cause: failures[0] }, + ) + } + } finally { + armed = false + listener.unsubscribe() + await live.cleanup() + await source.cleanup() + } } async function checkAuthorityCase( testCase: AuthorityCase, + reached?: ( + state: `release-invalidation` | `deferred-refill` | `reset-continuation`, + ) => void, + observeRejection?: (reason: unknown) => unknown, + observeRelease: (released: boolean) => boolean = (released) => released, ): Promise { const ledger: Array = [] const participants: Array> = [] @@ -561,6 +765,7 @@ async function checkAuthorityCase( const retryStartThrew = () => startThrewDuringRetry let startThrows = 0 let identityViolation: string | undefined + let requestErrorChecks = 0 // A request may fail, but the loader's own transition table must never // reject a move: that is a broken rule, wherever it surfaces. let invariantViolation: string | undefined @@ -591,6 +796,9 @@ async function checkAuthorityCase( let continuationOwed = false let invalidateNextPage = false let releaseFaults = 0 + let releaseInvalidations = 0 + let retirementInvalidationAt: number | undefined + let replayContinuationHandoffs = 0 let syncRepairs = 0 let invalidatingPages = 0 // The rank each continuation's cursor starts after, so a read of its @@ -754,6 +962,13 @@ async function checkAuthorityCase( releaseViolation ??= `request ${request.index} was released while a truncate replay was pending` } request.released = true + if (testCase.releaseInvalidates && releaseInvalidations === 0) { + releaseInvalidations++ + reached?.(`release-invalidation`) + retirementInvalidationAt = ledger.length + repairOwed = true + loader.invalidateSourceOrdering() + } if (testCase.releaseDisposes && !disposed) { disposed = true disposals++ @@ -799,7 +1014,8 @@ async function checkAuthorityCase( } finally { settlingChainMember = settling } - if (ledger.length === before) continuationOwed = true + if (ledger.length === before && request.issuedTruncates === truncates) + continuationOwed = true return } if (invalidateNextPage && method === `limited` && !authoritative) { @@ -863,7 +1079,12 @@ async function checkAuthorityCase( participants.push(result.catch(observe)) if (request) { participantOf.set(request, result) - result.catch((reason: unknown) => rejections.set(request, reason)) + result.catch((reason: unknown) => + rejections.set( + request, + observeRejection ? observeRejection(reason) : reason, + ), + ) } }, () => testCase.retryTimer, @@ -895,6 +1116,10 @@ async function checkAuthorityCase( let replaySettledAt = -1 let replayCoverage = new Set() const settleReplay = () => { + if (continuationOwed) { + replayContinuationHandoffs++ + reached?.(`deferred-refill`) + } truncatePending = false replaySettledAt = ++clock for (const request of ledger) { @@ -906,7 +1131,9 @@ async function checkAuthorityCase( ledger.filter((request) => request.fullSource && !request.released), ) if (replayCoverage.size > 0) replayAuthority = ledger.length - 0.5 - run(() => loader.settleFullSourceReplay()) + // Replay settlement is not a graph pass. It can finish retirement while + // joined work still owns a staged continuation; only a pass consumes it. + run(() => loader.settleFullSourceReplay(), false) } let unexpressibleBoundaries = 0 let joinedDeferrals = 0 @@ -965,7 +1192,7 @@ async function checkAuthorityCase( let sourceDeletes = 0 let timerRetries = 0 let shrinkingMoves = 0 - const run = (work: () => unknown) => { + const run = (work: () => unknown, consumesContinuation = true) => { const before = ledger.length try { const result = work() @@ -976,7 +1203,9 @@ async function checkAuthorityCase( } // A pass that issues nothing consumed or finished an owed continuation, // unless joined work deferred it. - if (ledger.length === before && !joinedPending) continuationOwed = false + if (consumesContinuation && ledger.length === before && !joinedPending) { + continuationOwed = false + } } const inflight = () => ledger.filter(({ outcome, hung }) => outcome === `inflight` && !hung) @@ -986,6 +1215,10 @@ async function checkAuthorityCase( const replayPending = () => truncatePending function truncate() { truncates++ + // Reset discards any staged continuation. A later settlement may stage a + // new one only for a request issued in this replay generation. + if (continuationOwed) reached?.(`reset-continuation`) + continuationOwed = false // A second truncate while a replay is pending, possibly one that failed. if (truncatePending) { stackedTruncates++ @@ -1025,7 +1258,13 @@ async function checkAuthorityCase( if (!ok && participantOf.has(request)) { const reason = rejections.get(request) if (rejections.has(request)) { - if (reason !== request.error && !injectedCleanup(reason)) { + if ( + request.issuedTruncates === truncates && + !request.replayed && + !isDisposed() + ) + requestErrorChecks++ + if (reason !== request.error) { identityViolation ??= `request ${request.index} failed with its own error, but its participant rejected with ${String(reason)}` } } else if ( @@ -1043,7 +1282,13 @@ async function checkAuthorityCase( settlingChainMember = false // An asynchronous settlement runs its continuation at once unless joined // work defers it; a settlement that issues nothing otherwise finished. - if (chainMember && ok && ledger.length === before && joinedPending) { + if ( + chainMember && + request.issuedTruncates === truncates && + ok && + ledger.length === before && + joinedPending + ) { continuationOwed = true } } @@ -1069,11 +1314,6 @@ async function checkAuthorityCase( ) const lastAuthoritativeSuccess = () => Math.max(lastAuthoritativeRead(), replayAuthority) - // A release callback's own error may replace a request's: cleanup failure - // is reported through the participant (ARCHITECTURE.md). - function injectedCleanup(reason: unknown) { - return reason instanceof Error && reason.message.startsWith(`release `) - } const checkHeld = (checkpoint: string) => { if (identityViolation) { throw new Error(`error identity ${checkpoint}: ${identityViolation}`) @@ -1547,6 +1787,16 @@ async function checkAuthorityCase( } await drain() checkHeld(`after the drain`) + // An invalidation delivered by adapter code during retirement is newer + // than the repair it interrupted. Once quiet, another authoritative start + // must have paid that debt (ARCHITECTURE, ordered recovery). The focused + // grammar below has no failure, disposal, or further truncate to decide it. + if (testCase.releaseInvalidates && retirementInvalidationAt !== undefined) { + checkRetirementProgress( + releaseInvalidations, + ledger.slice(retirementInvalidationAt).some(startsAuthority), + ) + } // A disposed loader owes nothing more; the drain above settled what it // had issued, and the dispose law has judged those settlements. if (!disposed) { @@ -1612,7 +1862,7 @@ async function checkAuthorityCase( const leaked = ledger.filter( (request) => request.outcome === `ok` && - !request.released && + !observeRelease(request.released) && request.index >= replayOwnedBefore && !replayCoverage.has(request) && request.index < authority, @@ -1800,6 +2050,9 @@ async function checkAuthorityCase( costChecks, hungRequests, offsetWindows: testCase.offset > 0 && ledger.length > 0 ? 1 : 0, + releaseInvalidations, + replayContinuationHandoffs, + requestErrorChecks, } } @@ -1830,11 +2083,61 @@ const authorityCampaigns = // the request ledger. The grammar is authorityCaseArbitrary, which the campaigns // and the calibration witnesses below read for coverage. declareOracle({ - check: [checkAuthorityCase], + check: [checkAuthorityCase, checkRetirementProgress, checkRetirementReceiver], grammar: [authorityCaseArbitrary], }) describe(`Ordered source request authority`, () => { + // Checker calibration changes only the recorded participant reason. Every + // case first executes the same legal current-failure history unmodified. + // Neither cleanup timing nor error text grants another value authority to + // replace that failure. The existing generated authority grammar contains + // these indexed/full-source, width, and throwing-release dimensions too. + it.each( + [false, true].flatMap((indexed) => + ([1, 2] as const).flatMap((limit) => + [false, true].flatMap((releaseFault) => + [false, true].map((requiresFullSource) => ({ + indexed, + limit, + releaseFault, + requiresFullSource, + })), + ), + ), + ), + )( + `rejects substituted participant errors across current failures: %j`, + async (axes) => { + const testCase: AuthorityCase = { + ...ordinaryScenario, + ...axes, + settledFirst: false, + syncRepair: false, + history: [{ type: `settle`, pick: 0, ok: false }], + } + const genuine = await checkAuthorityCase(testCase) + if (axes.releaseFault) expect(genuine.releaseFaults).toBeGreaterThan(0) + for (const replacement of [ + new Error(`release 0 failed`), + new Error(`another request failed`), + `release non-Error failure`, + ]) { + let substitutions = 0 + try { + await expect( + checkAuthorityCase(testCase, undefined, () => { + substitutions++ + return replacement + }), + ).rejects.toThrow(`error identity`) + } finally { + expect(substitutions).toBeGreaterThan(0) + } + } + }, + ) + // The model's strand trace: a repair supersedes the in-flight tie and // finishes; a newer invalidation's reaction is deferred; the old tie fails. it(`recovers a superseded tie that fails after its repair finished (generated-lane replay)`, async () => { @@ -2325,6 +2628,224 @@ describe(`Ordered source request authority`, () => { }, ) + // A replay's bookkeeping call is not a pass over the graph. In this trace, + // joined work staged a repair refill, then replay settlement retired older + // leases before the pass resumed that refill. It remains one repair chain. + // Source: https://github.com/TanStack/db/issues/2296 + it(`keeps a deferred repair refill in its chain after replay retirement`, async () => { + await checkAuthorityCase({ + indexed: false, + limit: 1, + settledFirst: true, + releaseFault: false, + syncRepair: true, + ...ordinaryScenario, + joined: true, + releaseTruncates: true, + history: [ + { type: `settle`, pick: 0, ok: true }, + { type: `settle`, pick: 0, ok: true }, + { type: `report`, react: false }, + { type: `move`, delta: -1 }, + { type: `joinedBusy` }, + { type: `settleNewest` }, + ], + }) + }) + + // The broad grammar already generated the source trace. This directed + // grammar makes its replay/continuation handoff frequent enough to measure, + // while index-free prefix size, local rows, joined deferral, synchronous + // answers, and release-triggered truncation vary independently. Without + // joined deferral there is no held continuation; without the release truncate + // there is no replay bookkeeping to distinguish from a graph pass. + const handoffProperty = `ordered-loader.replay-continuation` + const handoffCase = fc + .record({ + limit: fc.constantFrom(1 as const, 2 as const), + snapshotRows: fc.constantFrom(1 as const, 3 as const), + syncRepair: fc.boolean(), + joined: fc.boolean(), + releaseTruncates: fc.boolean(), + reset: fc.constantFrom( + `none` as const, + `before-settle` as const, + `after-settle` as const, + ), + }) + .map(({ reset, ...axes }): AuthorityCase => ({ + ...ordinaryScenario, + indexed: false, + settledFirst: true, + releaseFault: false, + ...axes, + history: [ + { type: `settle`, pick: 0, ok: true }, + { type: `settle`, pick: 0, ok: true }, + { type: `report`, react: false }, + { type: `move`, delta: -1 }, + { type: `joinedBusy` }, + ...(reset === `before-settle` ? [{ type: `truncate` as const }] : []), + { type: `settleNewest` }, + ...(reset === `after-settle` ? [{ type: `truncate` as const }] : []), + ], + })) + const handoffCampaigns = + authorityReplay.replayPath === undefined + ? ([`fixed`, `random`] as const) + : authorityReplay.replayProperty === handoffProperty + ? ([`replay`] as const) + : ([] as const) + it.each(handoffCampaigns)( + `preserves repair continuation identity across replay bookkeeping (%s campaign)`, + async (campaign) => { + const reach = createOracleReach( + handoffProperty, + [`deferred-refill`, `reset-continuation`] as const, + { minimum: 5 }, + ) + await fc.assert( + fc.asyncProperty(handoffCase, async (testCase) => { + reach.beginHistory() + await checkAuthorityCase(testCase, (state) => { + if (state !== `release-invalidation`) reach.reach(state) + }) + }), + campaign === `fixed` + ? { seed: 20261029, numRuns: oracleRuns(200) } + : oraclePropertyOptions(200, handoffProperty), + ) + reach.assertReached() + process.stderr.write( + `${handoffProperty} ${campaign} ${JSON.stringify(Object.fromEntries(reach.counts()))}\n`, + ) + }, + ) + + // An explicit invalidation in a release callback is new source information, + // unlike a replaced lease's own eviction deletes. It moves the ordering + // epoch while retirement still owns the repair slot. Both synchronous and + // asynchronous chains, indexed ties and unindexed growth must preserve it. + const retirementProperty = `ordered-loader.retirement-invalidation` + const retirementCase = fc + .record({ + indexed: fc.boolean(), + limit: fc.constantFrom(1 as const, 2 as const), + syncRepair: fc.boolean(), + joined: fc.boolean(), + needsMore: fc.boolean(), + }) + .map((axes): AuthorityCase => ({ + ...ordinaryScenario, + ...axes, + settledFirst: true, + releaseFault: false, + releaseInvalidates: true, + history: [{ type: `invalidate`, react: true }], + })) + const retirementCampaigns = + authorityReplay.replayPath === undefined + ? ([`fixed`, `random`] as const) + : authorityReplay.replayProperty === retirementProperty + ? ([`replay`] as const) + : ([] as const) + it.each(retirementCampaigns)( + `repairs an invalidation delivered during completed repair retirement (%s campaign)`, + async (campaign) => { + const reach = createOracleReach( + retirementProperty, + [`release-invalidation`] as const, + { minimum: 20 }, + ) + await fc.assert( + fc.asyncProperty(retirementCase, async (testCase) => { + reach.beginHistory() + await checkAuthorityCase(testCase, (state) => { + if (state === `release-invalidation`) reach.reach(state) + }) + }), + campaign === `fixed` + ? { seed: 20261029, numRuns: oracleRuns(200) } + : oraclePropertyOptions(200, retirementProperty), + ) + reach.assertReached() + process.stderr.write( + `${retirementProperty} ${campaign} ${JSON.stringify(Object.fromEntries(reach.counts()))}\n`, + ) + }, + ) + + // This driver receives the same law through real subscription and graph + // callbacks. Index availability, window width, and literal/Promise provider + // settlement vary; each completed history must reach the adapter callback. + const retirementReceiverCase = fc.record({ + indexed: fc.boolean(), + limit: fc.constantFrom(1 as const, 2 as const), + synchronous: fc.boolean(), + }) + // A shrink path belongs to one arbitrary and driver. The receiving grammar + // has its own identity so replay never applies a controlled history's path. + const retirementReceiverProperty = `ordered-loader.retirement-receiver` + const retirementReceiverCampaigns = + authorityReplay.replayPath === undefined + ? ([`fixed`, `random`] as const) + : authorityReplay.replayProperty === retirementReceiverProperty + ? ([`replay`] as const) + : ([] as const) + it.each(retirementReceiverCampaigns)( + `repairs retirement invalidation through a real subscription (%s campaign)`, + async (campaign) => { + const reach = createOracleReach( + retirementReceiverProperty, + [`real-subscription-release`] as const, + { minimum: 20 }, + ) + await fc.assert( + fc.asyncProperty(retirementReceiverCase, async (scenario) => { + reach.beginHistory() + await checkRetirementReceiver(scenario, () => + reach.reach(`real-subscription-release`), + ) + }), + campaign === `fixed` + ? { seed: 20261029, numRuns: oracleRuns(200) } + : oraclePropertyOptions(200, retirementReceiverProperty), + ) + reach.assertReached() + process.stderr.write( + `${retirementReceiverProperty} ${campaign} ${JSON.stringify(Object.fromEntries(reach.counts()))}\n`, + ) + }, + ) + + // Calibrate the shared no-leak judgment with a retained-lease observation. + // The real release callbacks still run; only their recorded result changes + // at comparison. This works for source and minified builds without reading + // private members. The same history first proves the unmodified path agrees. + it(`rejects retained successful acquisition observations after a covering repair`, async () => { + const testCase: AuthorityCase = { + ...ordinaryScenario, + indexed: true, + limit: 1, + settledFirst: true, + releaseFault: false, + syncRepair: false, + history: [{ type: `invalidate`, react: true }], + } + await checkAuthorityCase(testCase) + let substitutions = 0 + try { + await expect( + checkAuthorityCase(testCase, undefined, undefined, (released) => { + if (released) substitutions++ + return false + }), + ).rejects.toThrow(`no leak`) + } finally { + expect(substitutions).toBeGreaterThan(0) + } + }) + it.each(authorityCampaigns)( `holds, retires, and recovers requests under generated settlement orders (%s campaign)`, async (campaign) => { @@ -2336,6 +2857,10 @@ describe(`Ordered source request authority`, () => { let releaseFaults = 0 let syncRepairs = 0 let invalidatingPages = 0 + const identityReach = createOracleReach(authorityProperty, [ + `current-request-error`, + `cleanup-fault-current-request-error`, + ] as const) const later = { unexpressibleBoundaries: 0, joinedDeferrals: 0, @@ -2361,7 +2886,13 @@ describe(`Ordered source request authority`, () => { } await fc.assert( fc.asyncProperty(authorityCaseArbitrary, async (testCase) => { + identityReach.beginHistory() const observation = await checkAuthorityCase(testCase) + if (observation.requestErrorChecks > 0) { + identityReach.reach(`current-request-error`) + if (observation.releaseFaults > 0) + identityReach.reach(`cleanup-fault-current-request-error`) + } supersededFailures += observation.supersededFailures staleSettlementsAfterRepair += observation.staleSettlementsAfterRepair supersededFailuresAfterRepair += @@ -2389,6 +2920,10 @@ describe(`Ordered source request authority`, () => { // departure, a source delete, a backoff retry that issues a read, and // a window that shrinks. if (campaign !== `replay`) { + identityReach.assertReached() + process.stderr.write( + `${authorityProperty} identity ${campaign} ${JSON.stringify(Object.fromEntries(identityReach.counts()))}\n`, + ) expect(supersededFailures).toBeGreaterThan(0) expect(staleSettlementsAfterRepair).toBeGreaterThan(0) expect(supersededFailuresAfterRepair).toBeGreaterThan(0)