Skip to content
This repository was archived by the owner on Sep 25, 2026. It is now read-only.

Improve wildcard and phrase search for the 1.9 release - #200

Merged
ppodolsky merged 3 commits into
mainfrom
benchmark-and-optimize-hermes
Sep 23, 2026
Merged

ppodolsky merged 3 commits into
mainfrom
benchmark-and-optimize-hermes

Conversation

@ppodolsky

@ppodolsky ppodolsky commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

This change adds bounded whole-term wildcard queries and reduces phrase/conjunction and result-projection costs through the existing core readers and scorers. It also adds a reproducible Searchbench HTTP adapter, workload compatibility checks, and measured reports.

  • Share prefix/wildcard dictionary expansion with explicit scan, term and posting budgets; preserve logical document IDs after RGB reordering.
  • Batch posting intersections, reuse bounded position scratch, and apply conservative phrase score bounds while preserving exhaustive IDs, score bits and counts. New POS5/POS6 position certificates replace legacy position formats: older position indexes require rebuilding.
  • Bound the fast-field header directory to 256 checkpoints (3 KiB per reader), move owned envelope strings without allocations, and keep the in-place dispatch experiment opt-in. The corrected 32-client conjunction top-100 comparison improves 4.1% QPS and reduces CPU/request 3.8%; separate ID projection probes improve about 43%. Coverage remains 15 of 826 compatible queries, not the full suite.
  • Preserve current main's per-field tokenization, range cursor and broker-budget changes. Fix the existing Python README formatting failure in CI.

Validation: the final merged tree passes all nine full-harness stages (20260923T184501.534018Z-full), including 2,045 native tests (25 normally ignored), native-without-sync, portable core, API docs and real-server broker E2E. GitHub passes the WASM release build and all 40 JavaScript tests on commit 764b7c0b. Earlier byte/score/count audits, diagnostic-feature tests, and HTTP dispatch cancellation/panic/shutdown checks also pass. The benchmark report records exact binaries, memory, rejected experiments and the remaining performance gap. Both benchmark VMs are stopped.

Release: merge before dispatching publish.yml with version_bump=minor, producing 1.9.0 from 1.8.149. The workflow owns package version updates and tags.

…e-hermes

# Conflicts:
#	docs/README.md
#	docs/lexical-vertical.md
#	docs/search-performance-review.md
#	hermes-core/src/structures/fast_field/mod.rs
@ppodolsky
ppodolsky merged commit b43af70 into main Sep 23, 2026
10 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant