Repository navigation
Conversation
A jitexpr value source evaluated its expression once per doc. For an expression of a text input, e.g. `(REGEXP_EXTRACT service "^([^-]+)" 1u64)`, the result only depends on the term of the doc, and text inputs are dictionary encoded with typically few distinct terms per segment (service names, hosts, statuses). Evaluating the expression per doc made a terms aggregation over such a calculated field ~100x slower than over the physical column; on a CloudPrem searcher, the expression accounted for 91% of the CPU of such a query. Memoize the results of an expression of a single text input whose dictionary has at most 65,536 terms, per input term ord. The expression is evaluated on the first doc of each term, and later docs read the encoded result from two dense arrays allocated zeroed when the source is opened (9 bytes per term), plus a slot for docs without a value. Terms are only resolved for the term ords evaluated for the first time. jitexpr functions are deterministic, and `Str` results are interned in the order of the docs as before, so the values and term ords are unchanged. `to_f64` clears the memo, whose results are encoded for the previous result type. Other expressions keep the per-doc path. benches/jitexpr_agg_bench.rs (2M docs, 4 segments, 200 services): terms on (REGEXP_EXTRACT service ..) 140.6 ms -> 7.7 ms (18x) terms on (UPPER service) 305.7 ms -> 8.0 ms (38x) terms on (REGEXP_EXTRACT request_id ..) 1.441 s -> 1.444 s (unique terms, no memo) terms on physical service 1.08 ms -> 1.07 ms
- Replace `TermOrdMemo::get`/`set` by `get_or_eval`, and store the result for a doc without an input value in slot 0 (term ord `o` in slot `o + 1`) instead of a separate field. - Index the memo directly: a term ord out of the input dictionary is a corrupted column, which panics like the rest of the module. - Destructure the source once per block, and evaluate misses with a free `eval_single_input`, instead of a per-doc `memoized_result` method. - Share the cardinality tail of the two `load_block` paths, and implement `InputColumn::load_block` on top of `load_block_values`. - Drop the test-only memo threshold: the limit test builds a dictionary of `TERM_ORD_MEMO_MAX_NUM_TERMS + 1` terms, and the other tests exercise the memo for their small dictionaries. jitexpr_agg_bench: terms on (REGEXP_EXTRACT service ..) 7.7 ms -> 6.8 ms, on (UPPER service) 8.0 ms -> 6.8 ms; the other cases are unchanged.
Keep the PR focused on the memo; the benchmark numbers in the PR description were measured with it.
`TermOrdMemo` named the technique, not what it holds: the results of the expression per input term ord. Rename it `TermOrdResults`, the field `term_ord_results`, the limit `TERM_ORD_RESULTS_MAX_NUM_TERMS`, and the slot states `RESULT_UNKNOWN`/`RESULT_NULL`/`RESULT_VALUE`. No behavior change.
The slots of `TermOrdResults` are allocated once, when the source is opened, and never grow: store them in boxed slices rather than vectors. `vec!` of zeroes still allocates zeroed memory, and converting it does not reallocate. Also document why there is one slot more than terms: slot 0 holds the result for the docs without a value, and comes first so that a corrupted term ord is out of bounds instead of reading it.
- Replace `term_ord_results: Option<TermOrdResults>` by an `Evaluation` enum, `PerDoc` or `PerTermOrd(TermOrdResults)`: `None` meant per-doc evaluation, which the type did not say. - Size the `TermOrdResults` slices to the number of terms, and keep the result for the docs without a value in separate fields, instead of a slot 0 shifting every term ord by one. A term ord out of the dictionary is still out of bounds. `get_or_eval` handles both through the same state/value pair. No behavior change; jitexpr_agg_bench is unchanged.
65_536 rather than 1 << 16: the limit is a number of terms, not a bit width. No behavior change.
Replace the test-only `Evaluation::is_per_doc`/`is_per_term_ord` helpers with `matches!` at their call sites.
Remove the one-line accessors with a single caller: read the `var_type`, `column` and `first_values` fields directly (visible to the jitexpr value source module only), and fold `value_for_doc` into `value_of`, which now takes an `Option<u64>` and returns null for `None`. `InputColumn` goes from 11 to 6 methods. No behavior change.
Restore `InputColumn::var_type()`, and its call in `open_segment`.
Branch on `Evaluation` with a `match` in `load_block`, `eval_single_doc` and `to_f64`, instead of `if let Evaluation::PerTermOrd`, so that both strategies read as arms. No behavior change.
Mallets
force-pushed
the
mallets/jitexpr-term-ord-memo
branch
from
October 30, 2026 14:27
11d5c7b to
346c450
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A jitexpr value source evaluates its expression once per doc. For an expression of a text input, such as
(REGEXP_EXTRACT service "^([^-]+)" 1u64), the result only depends on the term of the doc, and text inputs are dictionary encoded with typically few distinct terms per segment (service names, hosts, statuses). Evaluating per doc made a terms aggregation over such a calculated field ~100x slower than over the physical column. On a production-sized CloudPrem searcher, the expression accounted for 91% of the CPU of such a query (39% of it inArcrefcount traffic fromRegex::captures), and 15-minute Log Explorer requests exceeded the 30s timeout.This PR memoizes the results of an expression of a single text input whose dictionary has at most 65,536 terms, per input term ord:
TermOrdResults::get_or_evalper doc.InputColumn::load_block_values/value_of).Strresults are interned in the order of the docs as before, so values and term ords are unchanged.to_f64clears the memo, whose results are encoded for the previous result type.Benchmark
An ad-hoc binggan benchmark, not included in this PR (2M docs, 4 segments, 200 services, single thread), against
9f9030a5f:(REGEXP_EXTRACT service "^([^-]+)" 1u64)(UPPER service)(REGEXP_EXTRACT request_id ..)(unique terms, no memo)serviceOn the CloudPrem searcher, the 15-minute group-by went from exceeding the 30s timeout to 5.4s, and the calculated field evaluation dropped from 91% to ~5% of the samples.
Tests
UPPER,REGEXP_EXTRACT,EQ,IS_NULL,IF) across repeated blocks andeval_single_doc, with the same interned terms in the same order;to_f64after a memoized block.cargo test --lib --features jitexprandcargo test -p jitexprpass. The limit test builds a dictionary ofTERM_ORD_RESULTS_MAX_NUM_TERMS + 1terms; the other jitexpr tests exercise the memo for their small dictionaries, and the equivalence test also disables it to compare with the per-doc path.The memo also applies to
eval_single_doc, whichJitExprPredicatecalls per doc: calculated predicates of a single text input benefit as well.Not in this PR: the per-call
Capturesallocation and regex pool contention inregexp_extract, which still matter for expressions that cannot be memoized.