Repository navigation
Description
A code review of Allow stored-field sorting and exact counts for full-text search identified a timing-dependent gap between full-text highlighting and final Table materialization. Runtime reproduction is now confirmed: three genuine-TTL tests fail at 476386d1e8b4d392a66ef055fd26f756add52ea9 because a promoted visible row lacks highlights; all three pass with the final-page fix.
At reviewed revision 2c75b0d1c837c72d515bcd9992cf93e7a67fb259, the full-text index validates current source versions and expiry, traces only the requested page and releases its reader lease before Table finishes materialization. Table checks expiry again during result transformation. A selected row can expire between those stages; discarding it can promote a retained tail row outside the traced window, leaving that returned row without the requested $highlights.
The root fix needs coordination between Table's final eligibility/pagination and fulltext's reader-scoped tracing. Preserve Harper's ownership of record eligibility and the wrapper/Tantivy ownership of matching and highlighting. Tracing every retained tail row would defeat page-only highlighting and is not the intended workaround.
Acceptance:
- Add a deterministic regression reproducing expiry between validation and final page materialization, including promotion of a retained tail row.
- Trace the final returned page while preserving the same-reader guarantee and page-only work bounds.
- Preserve cancellation, source-version validation and existing count/response contracts.
- Cover sorted and exact-count paths that retain rows beyond the highlighted window.
Parent qualification work: Qualify native full-text filtering and catalog-scale operation for v5.4. The closed Reuse full-text highlight tracing without an extra continuation reader addressed reader reuse; it does not track this final-page eligibility race.
Priority rationale: P2, a contained timing-dependent incomplete-metadata result; no demonstrated data loss, outage or authorization bypass. Effort: Medium, because the known cause spans Table pagination and reader-scoped tracing and needs a deterministic timing harness. No release is committed. Introducing version/PR remains unverified; PR 3846 is the discovery context, not an asserted origin.
Review-Deferral: 28af9a8071dc7073 severity=nit category=correctness
Comment generated by kAIle (GPT-6)
Review-Deferral: c2a8334918923b07 severity=nit category=correctness
This is an additional review fingerprint for the same expiry-driven highlight promotion described above, not a second defect. The final-page fix in Allow stored-field sorting and exact counts for full-text search addresses both fingerprints; this issue remains open until that implementation merges.
Comment generated by kAIle (GPT-6)
Implemented fix and verification
Allow stored-field sorting and exact counts for full-text search now runs final eligibility through Table before tracing the final page under the original native reader lease. It reuses request-local selected records so expiry/projection are not repeated after tracing. It does not trace retained tail/offset rows. Offset eligibility remains necessary; visible projection and source refresh share the execution budget. A stalled refresh returns 503 and releases the lease, rather than returning an unhighlighted page.
Ten additional page-preparation regressions cover real native batch tracing, concurrent isolation, nested relationship restrictions, onlyIfCached expiry messages, error/cancellation identity and unresolved source refresh with actual reader retirement. Final focused source and compiled suites each pass 181 tests with four existing pending tests; deployed REST/docs: 27 pass. These tests do not establish catalog/load qualification.
Comment generated by kAIle (GPT-6)
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity