fix(router): exclude PreferredModels turns from semantic cache

[rohith500]

Fixes #789.

Summary

cacheEligible already excludes requests where subsidyFactors (per-
installation subscription quota headroom) or SubscriptionOnlyFromContext
are in play, because both can change the scorer's winning model without
changing the semantic cache's isolation key (EffectiveKnobsHash). This PR
adds the same exclusion for PreferredModels, which has the identical
structural problem and was missing the gate.

Background / root cause

The semantic cache buckets responses by {format, clusterID, clusterVersion, knobsHash} (internal/router/cache/cache.go). knobsHash comes from
ComputeKnobsHash(alpha, speedWeight, outputCostRatio, expectedOutputTokens, perModelVerbosity) (internal/router/cluster/knobs.go) — a hash over the
routing knobs, not over anything that identifies which model actually won.

blendScoresV2 (internal/router/cluster/scorer.go) computes per-model
scores from those knobs, then adds two additional per-request adjustments
on top, inside the same loop:

if f, ok := subsidyFactors[m]; ok {
    scores[m] += subsidyMaxBonus * float32(1.0-f)
}
if b, ok := priorityBonus[m]; ok {
    scores[m] += b
}

priorityBonus is built from req.PreferredModels (rank-0 bonus ≈0.15,
decaying by rank, added once per top-P cluster — so it compounds and can
flip a genuinely close call, not just a tie; see
TestScorer_PreferredModelSoftNudge). Both subsidyFactors and
priorityBonus can change which model wins without changing
EffectiveKnobsHash, because the hash is computed from knobs alone,
before either adjustment is applied.

cacheEligible already accounted for this for subsidyFactors (and
separately, SubscriptionOnlyFromContext, for a related but distinct
reason — see the existing comment). It didn't account for it for
PreferredModels, introduced later in #521. That gap meant: two requests
with the same prompt/embedding/knobs but different PreferredModels could
produce different winning models, land in the same cache bucket, and a
cache hit would serve the wrong model's response body while
writeCachedResponse set x-router-model from the current (correct)
decision — a body/header mismatch, confirmed in #789 via a mechanism-level
test against the real Scorer and Cache (see the issue for the isolated
test and its output).

Fix

Mirrors the existing subsidyFactors gate exactly, at both call sites:

// internal/proxy/service.go, ProxyMessages (~line 2292)
cacheEligible := s.semanticCache != nil && !env.Stream() && decision.Metadata != nil &&
    externalID != "" && !bypassEval && !compactionHandoverRan &&
    !billing.SubscriptionOnlyFromContext(ctx) &&
    len(s.subsidyFactors(ctx, r.Header)) == 0 &&
    len(s.preferredModelsForRequest(ctx)) == 0   // <- added

Same addition at the OpenAI path (ProxyOpenAIChatCompletion, ~line 4220).
Comments above both sites updated to mention PreferredModels alongside
subsidy factors as a "score-perturbing input the cache key doesn't capture."

Why gate, not fold

Two existing patterns in this codebase address this class of gap:

• Gate (cacheEligible exclusion) — used by feat: subscription-aware routing (discount covered models by observed quota headroom) #495 (subsidy) and router: serve subscription-only below the overdraft floor instead of 402 #747
  (SubscriptionOnly).
• Fold (incorporate the perturbing input into the hash, e.g.
  mixModel) — used by feat(router): bandit foundation — propensity + reward logging, gated explorer #338 for the bandit explorer's non-argmax pick, and
  originally by feat(router): runtime-tunable routing knobs via v2 bundle format #214 for clusterVersion.

PreferredModels is structurally identical to subsidyFactors — both are
per-request maps added additively to scores[m] in the same loop, i.e.
score-perturbing inputs. The bandit explorer's case is different in kind:
it's a post-hoc override of an already-fixed argmax pick for online-learning
exploration, not a perturbation of the scoring function itself. Given that
structural match, gating (matching #495/#747) is the more consistent choice
here, and it's also the smaller, lower-risk diff — folding would require
either restructuring EffectiveKnobsHash's computation to happen after
blendScoresV2 resolves a winner (it currently runs before), or hashing the
full priorityBonus map, both larger changes than this PR's scope.

Trade-off: gating means any installation with a non-empty preferred_models
list gets zero semantic-cache benefit, full stop, rather than only losing it
on the specific requests where the preference actually changes the outcome.
Given preferred_models defaults to {} (opt-in, not the common case),
this seemed like the right trade for a minimal, safe fix — flagging here in
case there's a strong preference for the fold approach instead, since it's
a real trade either way.

Commits

1. bfbf829 — failing test reproducing the collision (confirmed failing
   for the right reason: cache hit + wrong body, not a compile/setup error)
2. dec5706 — the gate, at both call sites, plus comment updates

Testing

• New test (TestService_Cache_PreferredModelsBypasses) confirms: a
  request with non-empty PreferredModels now results in 2 real provider
  calls (no incorrect cache hit) instead of 1, and never reports
  x-router-cache: hit.
• Confirmed via the real service-layer request path (ProxyMessages,
  ProxyOpenAIChatCompletion), not just the cache package in isolation —
  cacheEligible is computed inline in service.go, so this is where the
  gate needed to be verified.
• Confirmed normal caching is unaffected: TestService_Cache_ HitShortCircuitsProvider (nil/absent preferences, the common case) still
  hits correctly; an explicit empty slice ([]string{}) also still hits.
• Confirmed preferredModelsForRequest(ctx) is nil/panic-safe: missing
  context key, empty slice, and a wrong-type context value (defensive type
  assertion) all resolve to len == 0 without panicking.
• Searched the whole repo for other semanticCache.Lookup/Store call
  sites — only the two fixed here exist. Gemini's proxy path doesn't use
  the semantic cache at all (cache.Format* only covers Anthropic/OpenAI).
  ProxyOpenAIResponses delegates to ProxyOpenAIChatCompletion and
  inherits the fix. /v1/route performs no cache I/O (dry-run).
• go vet ./internal/proxy/... clean; gofmt -l internal/proxy/service.go
  clean (no diff).
• Scoped suite green: ./internal/proxy/... ./internal/router/cluster/... ./internal/router/cache/..., verbose, -count=1.
• Full make check passes clean.

Known, deliberately unaddressed

• Stale-preference over-gating. The gate fires on "preference
  configured" (any non-empty preferred_models list on the installation),
  not "preference currently eligible for this request." An installation
  whose entire list is stale (references excluded/undeployed models —
  the same no-op case TestScorer_PreferredModelSoftNudge already covers
  at the scorer level) loses caching even though the scorer would ignore
  the stale entries entirely and routing is unaffected. This is safe
  (over-broad, not incorrect) and a minor hit-rate cost, not a correctness
  issue, so I left it out of this PR's scope rather than adding eligibility
  resolution logic here.
• ExcludedModels sibling gap. Same class of problem:
  installation.ExcludedModels can also change the scorer's winning model
  (by removing an otherwise-winning model from the eligible set) and isn't
  reflected in cacheEligible or EffectiveKnobsHash either. Didn't
  expand this PR to cover it since it's a separate root cause investigation
  and repro, but happy to open a follow-up issue if that's useful — let me
  know.

[greptile-apps]

PR author is not in the allowed authors list.

[devin-ai-integration]

Thanks for this, @rohith500 — really nice catch and a well-scoped fix. Reviewed against the repo conventions (root AGENTS.md + internal/proxy/AGENTS.md) and it conforms cleanly, no changes needed:

• Right layer / no new helper. The gate lives inline on *proxy.Service and reuses the existing preferredModelsForRequest(ctx) accessor rather than introducing a new one — matches "put logic in the package that uses it" and "don't add a helper unless needed in 3+ places."
• Mirrors the established pattern. Extending cacheEligible at both ProxyMessages and ProxyOpenAIChatCompletion is exactly the subsidyFactors / SubscriptionOnlyFromContext gate shape already in place — consistent, minimal, low-risk.
• No magic strings. The new test wires the service with providers.ProviderAnthropic per the "no bare provider/model literals" rule.
• Non-tautological test. TestService_Cache_PreferredModelsBypasses asserts real behavior (2 provider calls + no x-router-cache: hit) that breaks if the gate is removed, and the guard-comment style (guards #789 + the why) matches the accepted convention in this package.

The write-up on gate-vs-fold and the two deliberately-unaddressed follow-ups (stale-preference over-gating, the ExcludedModels sibling gap) is genuinely helpful context for the reviewer — appreciated. Approving on conventions; leaving the gate-vs-fold trade-off call to a maintainer since it's a product decision.