[kriszyp]

⊙ Problem

Queries that select, filter on, join through or sort by a @relationship read each related record synchronously. When one is not in memory (after a restart, on a new worker, under memory pressure), the worker blocks until storage returns, once per related record, and every other request on that worker waits.

🤔 Worth doing for all four query paths, without a latency measurement?

I think so: select alone would leave filters, joins and sorts blocking, and warm reads cost the same. Dropping those three would remove about 250 lines.

💡 Solution

Relationship reads in queries now use the async-capable read that top-level rows use, so a related record that is not in memory is read in the background.

• Select: a row is yielded only after its relationships load.
• Filter: conditions like children.label=x, including AND/OR groups and multi-hop paths.
• Join: a leading relationship condition.
• Sort: sort keys on related attributes.

Synchronous iteration (spread, asArray) and record.<relation> in application code stay synchronous. Warm queries cost the same as before.

⚖️ Alternatives

• Make record.<relation> async: breaks the documented synchronous accessor for application code.
• One MaybePromise path instead of sync + async forms: sync iteration and the vector-index pushdown need a boolean immediately.
• An async flag on the existing resolver: enough for select and sort, but filters and joins still need Promise-aware composition.
• Load selected related records through loadLocalRecord: about 0.3 µs slower per warm related row, and it changes db-read analytics and last-modified accounting.

Follow-ups: synchronous relationship conditions in the vector-index pushdown (harper#3587); one read per step when a query filters, sorts and selects on one relationship (harper#3606).

🔧 Changes

• resources/Table.ts: resolvers gain resolveKeys/loadEntries for many-to-one and array FKs, read through loadRelatedEntries, which keeps at most 16 related reads in flight once one goes to storage; select takes entries from loadEntries and observes reads it started if a later attribute throws, and a select with nested conditions, sort or limit reads related keys, then each record with async: true; transformToEntries uses maybeAsync filters only under async iteration; sort keys and grouping values are awaited, with relationship hops in getPathValue. When request teardown releases the read snapshot mid-query, the sorted iterator checks the read-snapshot guard after each awaited read, so on RocksDB it fails with the same expired-snapshot 503 a range read over that snapshot gives, and it closes its range iterator whenever next() rejects.
• resources/search.ts: relationship filters get a maybeAsync variant, and a filtered select's fromRecord stays sync under sync iteration; groups combine through matchesFilters; joinTo and joinFrom scan asynchronously through scanRelated with the query's read transaction, stopping the same way on a released snapshot; mapSettlingOnThrow observes pending matches.
• resources/DESIGN.md: the query-path rule beside the synchronous-accessor contract, the read window, teardown, and the one synchronous path left (harper#3625).
• utility/when.ts: a comment that restated when() removed.

🤔 Should a filter on a missing related record never match?

A child id that no longer resolves is skipped, so children.label != x does not match through it. Before, a to-many filter threw on it; a missing many-to-one never matched.

✅ Verification

• unitTests/resources/relationshipAsyncReads.test.js: counts blocking vs Promise-returning related reads for select, filter, join and sort, with one real cold RocksDB read (flushed SST, cleared record cache); the rest simulate cold reads. It also covers synchronous iteration, nested-constraint selects, the related-read window and request teardown mid-query. Each case fails on main or on an earlier commit of this branch.
• unitTests/resources/relationshipWarmReads.bench.js, everything cached: no difference from main beyond run-to-run noise, except a leading one-to-many join at about 65 ns per joined row.

Benchmark medians, 2000 parents, 30 runs each, three interleaved runs per side (ms)

Query	This branch	main
select child	5.44 / 5.24 / 5.40	5.27 / 5.21 / 5.75
select children	5.29 / 5.59 / 9.24	5.30 / 5.72 / 7.83
select parents (one-to-many)	11.74 / 12.20 / 11.62	11.96 / 11.88 / 12.82
filter child.name	4.72 / 4.70 / 4.72	4.74 / 4.63 / 5.03
filter children.name	4.31 / 4.79 / 4.27	4.12 / 4.19 / 4.31
sort child.name	3.45 / 5.46 / 5.50	5.37 / 5.35 / 5.89
sort children.name	4.59 / 4.73 / 4.59	4.52 / 4.53 / 4.66
leading join parents.score (154 rows)	0.25 / 0.25 / 0.26	0.24 / 0.24 / 0.24
plain filter	0.32 / 0.31 / 0.31	0.31 / 0.31 / 0.31

• 32 relationship, select and sort unit suites: 952 passing on RocksDB, 484 on LMDB; transaction and derived-index suites 137 and 80; relationship integration suites 89/89.
• PR QA: Do cold relationship queries keep one snapshot while related records change mid-query? — inconclusive @ 9c8e300 (no violation, cold == warm under concurrent writers; the oracle did not discriminate for the to-one filter or the leading join), qa-pr-harper-3615-f4d1e7b

_{🤖 Generated by Claude Opus 5.5 (Claude Code); posted via @kriszyp.}

Release target: main — performance change to query execution; no customer has asked for it on v5.3

Related PRs: #2155 overlaps (async bodies of joinTo/joinFrom), #3221 overlaps (select projection handler and its Promise chaining), #3279 overlaps (relationship filters, preserveForSelect and the projector), #3316 overlaps (relationship read permissions must also cover loadEntries and the async join reads), #3456 overlaps (select projection handler), #3571 overlaps (SKIP handling and cancellation in sorted iteration), #3605 overlaps (per-row work in transformEntryForSelect), #3610 overlaps (new loaders return residency stubs as the synchronous resolvers do), #3619 overlaps (cross-database fix must reach loadEntries and the async join reads), 53 others independent
Complexity: complicated
Framing-Verdict: chosen-approach-sound (e83832442c09)

Deferred findings

• df07c8ab76a9db41 major performance — resources/search.ts — Custom-index pushdown evaluates relationship siblings synchronously, blocking per cold related read — filed Vector-index pushdown evaluates relationship conditions synchronously, blocking on cold related reads #3587
• bfdbc83f2d11f0cf major performance — resources/search.ts — Async relationship filter leaf still reaches blocking adaptive inverse-key collection — filed Relationship filter's adaptive id-set switch still reads related records synchronously #3625
• 8dac330643cbe8cf nit performance — resources/Table.ts — Filter, sort and select on the same relationship each read the related row — filed A query that filters, sorts and selects on one relationship reads each related record once per step #3606
• aaa911f43627c71c nit correctness — resources/DESIGN.md — .to accessor asArray can return a Promise on caching tables — filed One-to-many relationship property can yield Promises when the related table is a caching table #3588
• ef49aa00280e2e58 nit performance — resources/search.ts — Async fromRecord discards loaded entries, so select re-reads matching related rows — filed A query that filters, sorts and selects on one relationship reads each related record once per step #3606
• b3ecc55fe7cf0358 nit correctness — resources/search.ts — toMany related predicates are called without the entry argument — NOT FILED

Late Grok findings

3 surviving (highest major) — Grok reviewed d3f0201

• major — resources/Table.ts:6902 — When an index provides the first sort key and sort.next exists, rows are grouped by comparing groupingValue !== lastGroupingValue. For a relationship path such as ['child','name'], getAttributeValue now returns a Promise whenever the related read misses the cache (`resources/Table.ts:10161-1
• minor — resources/search.ts:1403 — The async joinTo reads related records with store.get(...) without passing the query's read transaction, and the same applies to joinFrom at resources/search.ts:1518-1519. Both then await between rows. A commit landing during that await can change a related row that has not been read yet. A
• minor — resources/Table.ts:10150 — sort: { attribute: 'child' } sorts on the relationship name itself rather than a path into it. That case still calls the resolver's synchronous form (getSync/getEntry), so each cold related row blocks the worker during the sort. This contradicts the invariant this PR adds at `resources/DESIGN.

Not standing: none. Grok ran 19m (19m from launch to report); the fold adjudication (cursor-grok@d3f0201b94c2-767c23e1, claude) took 1m.

Late Grok findings

5 surviving (highest minor) — Grok reviewed 24c6106

• minor — resources/Table.ts:10256 — Relationship reads that are already in flight continue after the query is returned or its read snapshot is released. The nested-hop continuation at 10260 starts the next hop's read, loadRelatedEntry (10223-10229) has no lifetime check, and loadRemainingRelatedEntries checks the snapshot only at
• nit — resources/search.ts:1530 — The scan's upstream iterator.return() is not awaited: the scan settles before cleanup finishes. The async joinTo return() (1443-1446) only sets scan.closed, so the upstream iterator is closed only after a pending next() settles. The async joinTo iterator has no throw(). Scope: in-scope
• nit — resources/search.ts:1538 — In someRelatedMatch, a toMany related predicate gets match(value) with no entry argument. The sync sibling at 1872 (subObject.some(nextFilter)) passes the array index instead, so main already behaves this way. Scope: pre-existing — Convergence-Class: pre-existing
• nit — unitTests/resources/relationshipAsyncReads.test.js:385 — No test covers return() while a scan or sort collection is pending, a related read or batch in flight at teardown, or the sync joinTo path. These are cancellation-path gaps, not security branches. Scope: in-scope — Convergence-Class: previously-adjudicated
• nit — unitTests/resources/relationshipAsyncReads.test.js:76 — Remaining comment narration: :76 restates the depth guard beside it. resources/Table.ts:10183 and :68 are borderline. Most of the earlier-listed comments are gone. The ones left (Table.ts:6841-6843, 6983, 10198-10199, 10214; search.ts:1479, 1498, 1531) explain constraints and sho

Not standing: none. Grok ran 14m (14m from launch to report); the fold adjudication (cursor-grok@24c6106f59a3-d268f6fb, claude) took 2m.

_{Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; blocked=claude(fallback)(out-of-budget); declined=muse; rounds=13; full=5 @ fd30606}

_{Review-Attention: deep ~85m (critical: Table.ts, search.ts; decisions: batch-vs-window-related-reads, per-row-cap-not-global, direct-store-related-reads, released-snapshot-is-503, do-less-alternative) @ fd30606}

[gemini-code-assist]

Code Review

This pull request introduces asynchronous relationship reads to the database, allowing for non-blocking access to related records. The changes update core query logic in Table.ts and search.ts to support asynchronous iteration, filtering, and joining, while adding new test and benchmark suites. Feedback identifies a critical this binding issue in a callback, a potential TypeError when processing synchronous iterables, and the need for improved null-safety when loading related entries.