Skip to content

fix(cluster): computeDialCalibration ignores AlphaFloor — QualityBias dial dead zone on floored bundles #779

Description

@rohith500

Summary

computeDialCalibration records QualityBias mix-change breakpoints by sweeping unfloored uniform alpha, but live routing applies alpha[i] = max(dialToAlpha(t), floor[i]) via applyDialAlpha. On current floored bundles (v0.72–v0.74, latest = v0.73), roughly the first half of the dial produces an identical model mix / projected cost — the same class of “50% looks like 20%” failure that #479 fixed, reintroduced when #501/#515 added floors without retargeting the calibration sweep.

Why it matters

  • Routing correctness / cost-quality UX on the customer-facing QualityBias dial (dashboard + x-weave-routing-*).
  • Demonstrated on committed production artifacts, not a synthetic edge case.
  • Availability unaffected; quality and spend mix stop responding to dial travel until ~t=0.5.

Where

  • internal/router/cluster/scorer.gocomputeDialCalibration (unfloored sweep)
  • internal/router/cluster/scorer.goapplyDialAlpha / dialToAlpha (floored consumer used by Route and RoutingDistribution)

Reproduction (v0.73)

Same 21-point RoutingDistribution grid used by #479’s regression tests:

Bundle alpha_floor Breakpoints with α ∈ (0, minFloor) Adjacent identical mixes Cost at dial 0 / 0.2 / 0.5
v0.67 none 0 1 0.00010 / 0.00033 / 0.00116 ✅
v0.70–0.71 0.30 1 4 mild degradation
v0.72–0.74 (latest) 0.30 9 11 (max run 10) 0.00187 / 0.00187 / 0.00187

Counterfactual: recompute calibration with per-cluster max(a, floor[i]) (same policy as applyDialAlpha) → breakpoints 30 → 14, zero sub-floor breakpoints.

Historical regression mechanism

  1. fix(routing): calibrate quality dial against real mix crossovers #479 — calibrated dial against real mix crossovers (computeDialCalibration / dialToAlpha). Regression tests (TestRoutingDistribution_NoDeadZone, mid-vs-low cost) pin on v0.67, which ships no alpha_floor.
  2. fix(routing): per-cluster alpha_floor so the price/quality dial keeps every cluster at its best-for-budget model #501 — added alpha_floor via applyDialAlpha = max(dialToAlpha(t), floor[i]). Did not update the calibration sweep. Reviews did not mention computeDialCalibration.
  3. fix(routing): gate agentic pool to harness-capable models so the price dial can demote Opus #515 — relaxed agentic floors 0.88 → 0.30 and moved harness guard to the candidate pool. Calibration still unfloored.
  4. v0.72+ — roster/fable changes multiplied unreachable low-α breakpoints (1 → 9); dead zone became customer-visible on latest.

Recommended fix

Share a helper that turns a raw uniform alpha into the effective floored per-cluster vector, and use it from both computeDialCalibration and applyDialAlpha.

Regression coverage:

  • Synthetic heterogeneous-floor test (different floors per cluster).
  • Frozen floored-bundle regression (v0.73 or successor): no long identical-mix run; mid dial materially pricier than low.
  • Backward-compat: bundles without alpha_floor keep current calibration behavior.
  • Before/after RoutingDistribution / cmd/routing-report --quality-bias report.

Do not keep NoDeadZone / mid-vs-low guards only on v0.67 — that is how this regressed silently.

Confidence

High — reproduced on committed artifacts; counterfactual confirms root cause; no open issue tracks this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions