Filter-cost sort in stats.go uses a non-strict comparator (undefined tie order; blocks sort.Slice→slices migration)

[fengttt]

Summary

The filter-cost sort in pkg/sql/plan/stats.go (sortFilterListByStats / the Node_TABLE_SCAN branch) orders a scan's FilterList with a non-strict comparator:

sort.Slice(node.FilterList, func(i, j int) bool {
    cost1 := estimateFilterWeight(node.FilterList[i], 0) * node.FilterList[i].Selectivity
    cost2 := estimateFilterWeight(node.FilterList[j], 0) * node.FilterList[j].Selectivity
    return cost1 <= cost2   // <-- returns true for equal costs
})

cost1 <= cost2 is not a valid strict weak ordering (it reports "less" in both directions for equal costs). Consequences:

1. The relative order of equal-cost filters is undefined — it is an artifact of sort.Slice's (unstable) algorithm interacting with the non-strict predicate, not a defined priority.
2. That undefined order is observable, because downstream consumers depend on it:
   • it selects which distance predicate an IVF mode=pre index probes with (→ wrong results, see IVFFLAT 'mode=pre' drops extra distance predicates (residual filter not applied) → wrong results, order-dependent #25639);
   • it selects which value a runtime "not a valid bool expression" error reports (test/distributed/cases/operator/is_not_operator.sql).
3. It cannot be migrated to a strict slices.SortFunc comparator without changing behavior. This is why PR perf: replace stdlib sort.Slice with slices pkg to eliminate heap escapes #25616 (sort.Slice → slices) had to leave exactly this one site on sort.Slice.

Why it can't just be made strict today

Switching to a strict comparator (e.g. cmp.Compare(cost1, cost2), stable or not) is the desirable end state, but doing so changes the equal-cost tie order, which currently regresses correctness via #25639: vector/vector_ivf_mode then returns rows past the distance bound (7 rows instead of 3). So this cleanup is effectively blocked by #25639 — the order must stop being correctness-relevant before it can be redefined.

Proposed fix (once #25639 is resolved)

• Give the filter-cost sort an explicit, deterministic, valid strict weak ordering (strict cmp.Compare on cost, plus an explicit tie-breaker if a defined priority among equal-cost filters is desired), and
• migrate it to slices.SortFunc/SortStableFunc to match the rest of the codebase and drop the sort.Slice interface{} boxing.

Consumers of FilterList order should not rely on tie ordering for correctness (that is the #25639 half); this issue is the planner half — make the order well-defined in the first place.

Relationship to #25639

Related, not duplicate. #25639 is the IVF index dropping residual predicates (order-sensitive consumer). This issue is the planner producing an ill-defined filter order (non-strict comparator). Fixing #25639 unblocks the strict/deterministic cleanup here; fixing this makes the emitted order well-defined. Both are needed for a fully robust result.

[fengttt]

Fixed in PR #25616 (commit edd3c9f). With #25639 resolved, the IVF scan is no longer order-sensitive, so the filter-cost sort no longer needs the fragile sort.Slice(<=) order. It now uses slices.SortStableFunc with a strict cmp.Compare — a valid, deterministic strict weak ordering where equal-cost filters keep their as-written relative order. Will auto-close on merge.