Repository navigation
Description
Outcome
Let callers require a text match while rewarding optional matches, and adjust supported clause/field weights per request without rebuilding the index. For example, require a product match and reward a phrase match in its title.
Ownership and dependency
Parent: Epic: Integrate schema-native full-text search into Harper. Depends on the scoring increment in Extend typed queries with optional scoring clauses, boosts, regex, and similar-document search. The wrapper owns typed query semantics and native scoring. Harper owns public query validation, planning, authorization, source validation and result projection.
Acceptance
- Define Table.search and REST representations that map to the released wrapper's typed boost and required/optional scoring contracts. Preserve existing query defaults.
- Define optional-only queries, minimum optional matches and nested Boolean behavior explicitly. With a required clause, an unmatched optional scoring clause must not remove an otherwise eligible row unless the request explicitly requires it.
- Distinguish persisted index-definition field weights from request-level field overrides and clause boosts; specify precedence/composition. Do not promise per-query field weights until the wrapper exposes them.
- Validate supported finite weight ranges, expression depth, term/clause capacity and request budgets before native execution. Apply host query policy consistently across workers.
- Authorize every referenced text field, including fields used only to influence ranking, before native execution. Preserve source-version/expiry checks and correct refill after rejected candidates.
- Define
$score, deterministic tie order, pagination, counts and highlight behavior. Scoring-only clauses must not accidentally change membership or totals. - Test Table.search and deployed REST with positive/negative relevance examples, defaults, ties/pages, restricted scoring fields, invalid weights and limit exhaustion. Add release compatibility and documentation examples.
Boundaries and planning
Regex, similar-document retrieval, metadata-derived business ranking and BM25/vector fusion are separate increments. This issue does not require raw query-parser syntax.
Proposed feature; API and release are not committed. Priority P3: new relevance controls under a P3 epic, with no established customer blocker or committed release. Effort High: native contract adoption plus public query/planner, permission and paging integration. No customer-specific or security context.
Draft implementation progress (2026-10-31)
Draft companions: Add full-text boosts and optional scoring conditions, Add query boosts and optional full-text scoring, and Document full-text boosts and optional scoring.
Merge gate: release the compatible fulltext wrapper and update Harper's production dependency pin before merging the Harper implementation. These drafts do not commit a release version or add per-query field-weight overrides.
The host precheck deliberately bounds descriptor counts coarsely. The native compiler enforces the expanded clause budget. The real-native boundary test uses one required single-field term: 127 optional single-field terms compile to 256 clauses and succeed; 128 compile to 258 clauses and return HTTP 400. Both shapes pass the coarse host precheck. This qualification addresses the correctness and test-coverage markers below; it does not claim all compiled-capacity failures are rejected before native execution.
Coordinate overlapping changes with Isolate permissions when reusing nested relationship selectors and Allow stored-field sorting and exact counts for full-text search. The maintainability marker records merge coordination, not a new defect.
Review-Deferral: b77412c2c0723088 severity=none category=correctness
Review-Deferral: 4550e3d8d6a7afc1 severity=none category=test-coverage
Review-Deferral: 1806ea835e950478 severity=nit category=maintainability
Parallel release and merge boundaries
Expose fulltext operation telemetry and latency histograms claims ABI 14/query API 7 plus telemetry API 1; the scoring wrapper claims ABI 14/query API 8. Reconcile these contracts before release. If telemetry's ABI 14 ships first, scoring needs a fresh ABI and this Harper compatibility pair must change with its production pin/lock. The scoring draft deliberately rejects telemetry-only 14/7. Detailed native ownership is tracked in Extend typed queries with optional scoring clauses, boosts, regex, and similar-document search.
When merging Allow stored-field sorting and exact counts for full-text search, retain the scoring-aware boost/optional walkers and capability, coverage and cancellation fixes in fullTextQueryIndex.ts. Define and document how explicit stored-field ordering interacts with scoring. Do not resolve conflicts by replacing the whole reader with either branch's version.
Configured-binding close coordination
Source trace at Add full-text boosts and optional scoring conditions, revision a53592d9: a previously admitted reader operation can finish configured-binding/runtime-info loading after close begins and then open a reader. This is preexisting work, not a scoring-response regression. Reader initialization records the pending operation; retirement waits for it and retires/closes the resulting reader. Subsequent acquisition checks the closed state. The trace establishes neither a reusable post-close handle nor a handle leak. Close/reset retain their bounded drain-timeout behavior.
A follow-up may avoid the redundant open by rechecking closed state after binding readiness. This is an optimization opportunity; no runtime reproduction or new defect is claimed here.
Review-Deferral: 7eefabe7d730624e severity=nit category=concurrency
Comment generated by kAIle (GPT-6)
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity
Reconcile cold scoring capability timeout with fixed reader bound
Add full-text boosts and optional scoring conditions (#3847)