[PSeitz]

This fixes a correctness issue in composte aggregation. But instead of fixing composite, I replaced it with the multi terms aggregation.

The fixed issue is (test is composite_aggregation_deduplicates_mapped_source_keys)

  Example:
  - Prices: [12, 15]
  - Histogram interval: 10
  - Tags: ["x", "y"]

  Without deduplication after mapping, composite generates:

    (10, x), (10, y), (10, x), (10, y)

Summary

Run composite aggregations through the multi-terms collector so both share block decoding, tuple expansion, bucket counting, and child-aggregation buffering. This removes the separate recursive composite collector and consolidates the collection logic.

Changes

• Translate composite term, histogram, and date-histogram sources into ordered tuple components, with a page-sized bucket map and after-prefix pruning.
• Deduplicate mapped values per document before expanding tuples, so values that land in the same histogram bucket contribute once.
• Specialize tuple keys for each supported source count (1–16) to avoid carrying unused components.
• Account for tuple-building buffers in aggregation memory limits.
• Add regression tests for missing-value ordering, pagination prefixes, mapped-key deduplication, and nested counts and child metrics. Add a multivalue composite benchmark with deletions and a sum child aggregation.

Benchmark

Composite benefits from the optimized multi_terms aggregation, but there are edge cases where it consumes more memory.

full
multi_terms_vs_nested
nested_terms_status_and_zipf_1000                 Memory: 245.4 KB             Avg: 3.8532ms (-5.85%)      Median: 3.8411ms (-5.96%)      [3.8189ms .. 3.9468ms]
multi_terms_status_and_zipf_1000                  Memory: 168.0 KB (-9.00%)    Avg: 1.3612ms (-5.63%)      Median: 1.3576ms (-3.88%)      [1.3449ms .. 1.3973ms]
nested_terms_status_and_zipf_1000_with_missing    Memory: 245.5 KB             Avg: 3.8495ms (-6.72%)      Median: 3.8428ms (-6.10%)      [3.8176ms .. 3.9716ms]
multi_terms_status_and_zipf_1000_with_missing     Memory: 168.1 KB (-8.99%)    Avg: 1.3615ms (-4.50%)      Median: 1.3555ms (-3.08%)      [1.3497ms .. 1.4266ms]
nested_terms_zipf_1000_and_status                 Memory: 434.7 KB             Avg: 11.4091ms (-7.15%)     Median: 11.3840ms (-6.31%)     [11.2891ms .. 11.6525ms]
multi_terms_zipf_1000_and_status                  Memory: 168.9 KB (-8.74%)    Avg: 1.3297ms (-2.35%)      Median: 1.3214ms (-2.43%)      [1.3075ms .. 1.4021ms]
nested_terms_many_and_zipf_1000                   Memory: 38.2 MB              Avg: 54.6546ms (-4.03%)     Median: 53.2242ms (-2.33%)     [51.3944ms .. 65.6167ms]
multi_terms_many_and_zipf_1000                    Memory: 31.9 MB (-0.00%)     Avg: 15.4064ms (-1.50%)     Median: 15.3783ms (-0.72%)     [15.2331ms .. 15.7382ms]
nested_terms_many_and_zipf_1000_and_status        Memory: 146.9 MB             Avg: 130.8759ms (-5.19%)    Median: 128.8879ms (-1.59%)    [123.6916ms .. 148.9116ms]
multi_terms_many_and_zipf_1000_and_status         Memory: 53.5 MB (-0.00%)     Avg: 21.4539ms (-0.81%)     Median: 21.3548ms (-0.33%)     [21.1454ms .. 22.6923ms]
nested_terms_status_and_zipf_1000_avg_sub_agg     Memory: 1.6 MB               Avg: 12.3954ms (+1.95%)     Median: 12.3624ms (+2.49%)     [12.1219ms .. 13.1284ms]
multi_terms_status_and_zipf_1000_avg_sub_agg      Memory: 618.6 KB (-0.02%)    Avg: 8.9847ms (+3.17%)      Median: 8.9973ms (+4.06%)      [8.6924ms .. 9.2979ms]
multi_terms_vs_nested_with_1%_term_query
nested_terms_status_and_zipf_1000                 Memory: 111.9 KB             Avg: 0.1555ms (+2.54%)     Median: 0.1556ms (+2.99%)    [0.1513ms .. 0.1608ms]
multi_terms_status_and_zipf_1000                  Memory: 70.9 KB (-18.98%)    Avg: 0.0792ms (+1.51%)     Median: 0.0788ms (+1.09%)    [0.0774ms .. 0.0813ms]
nested_terms_status_and_zipf_1000_with_missing    Memory: 112.0 KB (-0.34%)    Avg: 0.1558ms (+2.22%)     Median: 0.1549ms (+1.94%)    [0.1515ms .. 0.1643ms]
multi_terms_status_and_zipf_1000_with_missing     Memory: 71.2 KB (-18.93%)    Avg: 0.0799ms (+1.84%)     Median: 0.0799ms (+1.87%)    [0.0784ms .. 0.0820ms]
nested_terms_zipf_1000_and_status                 Memory: 343.6 KB             Avg: 0.2786ms (+1.81%)     Median: 0.2790ms (+2.50%)    [0.2725ms .. 0.2839ms]
multi_terms_zipf_1000_and_status                  Memory: 72.1 KB (-18.11%)    Avg: 0.0791ms (-8.66%)     Median: 0.0786ms (-2.64%)    [0.0775ms .. 0.0819ms]
nested_terms_many_and_zipf_1000                   Memory: 2.7 MB               Avg: 0.8286ms (-12.36%)    Median: 0.8294ms (-4.31%)    [0.8042ms .. 0.8540ms]
multi_terms_many_and_zipf_1000                    Memory: 445.4 KB (-0.02%)    Avg: 0.5501ms (-11.70%)    Median: 0.5492ms (-3.02%)    [0.5372ms .. 0.5677ms]
nested_terms_many_and_zipf_1000_and_status        Memory: 4.1 MB               Avg: 1.3241ms (-16.15%)    Median: 1.3202ms (-3.55%)    [1.2866ms .. 1.3697ms]
multi_terms_many_and_zipf_1000_and_status         Memory: 446.0 KB (-0.09%)    Avg: 0.5839ms (-11.29%)    Median: 0.5817ms (-2.74%)    [0.5694ms .. 0.6267ms]
nested_terms_status_and_zipf_1000_avg_sub_agg     Memory: 1.1 MB               Avg: 0.2954ms (-7.30%)     Median: 0.2947ms (-7.43%)    [0.2899ms .. 0.3017ms]
multi_terms_status_and_zipf_1000_avg_sub_agg      Memory: 224.8 KB (-0.16%)    Avg: 0.1416ms (-4.14%)     Median: 0.1413ms (-3.76%)    [0.1385ms .. 0.1475ms]
range_and_histogram
composite_term_few                                Memory: 23.6 KB (+10.61%)    Avg: 3.7385ms (-72.08%)    Median: 3.7399ms (-71.50%)    [3.6685ms .. 3.8147ms]
composite_term_many_page_1000                     Memory: 378.3 KB (+1.10%)    Avg: 6.6543ms (-60.04%)    Median: 6.6438ms (-59.71%)    [6.5373ms .. 6.8737ms]
composite_term_many_page_1000_with_avg_sub_agg    Memory: 2.3 MB (-2.66%)      Avg: 7.4115ms (-56.91%)    Median: 7.3779ms (-56.52%)    [7.2072ms .. 7.9410ms]
composite_histogram                               Memory: 299.8 KB             Avg: 4.8339ms (-71.83%)    Median: 4.8257ms (-71.64%)    [4.7362ms .. 5.0911ms]
composite_histogram_calendar                      Memory: 10.3 KB (+50.82%)    Avg: 9.7389ms (-68.60%)    Median: 9.7387ms (-67.97%)    [9.5689ms .. 10.0407ms]

composite_multivalue
16_values_per_field_page_256_with_sum_and_deletes    Memory: 44.4 MB (+844.14%)    Avg: 15.9355ms (+7.43%)    Median: 15.7627ms (+5.17%)    [15.3864ms .. 17.3827ms]

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-29T09:41:48.345457Z	630d00f	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93e9ea4c92

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Deduplicate documents before merging decoded composite buckets

When a composite terms source has two encoded values in one document that decode to the same public key (for example, a JSON multivalue [-1, -1.0], whose I64 and F64 forms both resolve to I64(-1)), the new collector creates separate internal buckets and reaches this occupied branch for both. Adding the bucket counts treats them as separate documents, so the result reports doc_count: 2 and merges child-aggregation data twice instead of counting the document once; the values need to be canonicalized/deduplicated per document before counting rather than summing overlapping encoded buckets here.

Useful? React with 👍 / 👎.