Gemini SWITCH never gets bounded-cost handover — ProviderSummarizer rejects FormatGemini

[rohith500]

TL;DR

On a planner SWITCH for a native Gemini request (ProxyGeminiGenerateContent), the handover summarizer always fails before any upstream call because ProviderSummarizer builds its request via PrepareAnthropic, which has no FormatGemini case. The orchestrator correctly falls back to full history, so the turn is safe — but the bounded-cost rewrite ([summary, latestUser]) that Anthropic and OpenAI SWITCH turns get never runs. Every Gemini SWITCH therefore pays the full prior contents at the new model's input rate. Live-verified end-to-end below.

Same under-wiring class as #730 / #731 (Gemini surface missing a sibling that Anthropic/OpenAI already have), different subsystem (handover / summarizer ingest).

---

Summary

Cache-aware turn routing on SWITCH is supposed to:

1. Call handover.Summarizer to produce a short prose summary.
2. On success, handover.RewriteEnvelope(env, summary) → [summary, latestUser].
3. On failure/timeout, keep full history (deliberate; never silent TrimLastN).

Steps 1–2 work for Anthropic and OpenAI ingress because buildSummaryRequestBody → env.PrepareAnthropic(...) accepts those formats. Native Gemini envelopes hit:

// internal/translate/emit_anthropic.go
default:
    return providers.PreparedRequest{}, fmt.Errorf("unsupported source format for Anthropic emit: %d", e.format)

So every Gemini SWITCH logs a summarizer failure and sets handover_fallback_to_full_history=true. RewriteEnvelope is never called. Planner SWITCH itself still happens (EV / tier / cold-pin levers); only the cost-bounding rewrite is missing.

This is not a silent correctness 400 today (full-history fallback preserves functionCall/functionResponse pairing). It is a structural cost gap on every Gemini SWITCH, plus warn-level log noise.

---

Expected vs Actual

Expected

For a pinned Gemini session where the planner decides SWITCH to another Google model (e.g. gemini-3.1-pro-preview → gemini-3.1-flash-lite-preview):

1. Summarizer ingests the Gemini envelope (via Gemini→Anthropic emit for the haiku summarizer, or a Gemini-native summarizer).
2. On success, RewriteEnvelope rewrites contents to [model summary, latest user].
3. Upstream Google sees a short bounded body on the switch turn.
4. Telemetry shows handover_invoked=true and handover_fallback_to_full_history=false.

This matches Anthropic (ProxyMessages) and OpenAI (ProxyOpenAIChatCompletion) SWITCH behavior.

Actual (live-confirmed)

1. Planner SWITCH still fires (reason=ev_positive, pin tier switch_ev_positive).
2. Summarizer fails immediately at prepare — no Anthropic upstream call.
3. Log:

   Handover summarizer failed; preserved full history instead
   err=build handover request: prepare anthropic body: unsupported source format for Anthropic emit: 2
   

4. Forwarded body keeps the entire prior contents (including tool turns).
5. Telemetry: handover_invoked=true, handover_fallback_to_full_history=true.

---

Exact mechanism

Call chain on Gemini SWITCH (pinFound && !prefixBroken, summarizer wired, tenant boundary allows the call):

Step	Where	What happens on Gemini
Planner SWITCH	internal/proxy/turnloop.go	Runs normally (OutcomeSwitch)
Summarizer.Summarize	internal/proxy/handover.go ProviderSummarizer.summarize	Calls buildSummaryRequestBody
Prepare	buildSummaryRequestBody → env.PrepareAnthropic	Errors: unsupported FormatGemini
Fallback	turnloop.go switch path	FallbackToFullHistory=true; does not call RewriteEnvelope
Emit / dispatch	internal/proxy/gemini.go	PrepareGemini same-format passthrough of full history

Root reject:

// internal/translate/emit_anthropic.go — PrepareAnthropic
switch e.format {
case FormatOpenAI:    // OK
case FormatAnthropic: // OK
default:
    return ..., fmt.Errorf("unsupported source format for Anthropic emit: %d", e.format)
}

FormatGemini is that default. OpenAI is accepted via buildAnthropicFromOpenAI, which is why OpenAI SWITCH can bound history with the same Anthropic-haiku summarizer.

RewriteEnvelope is only invoked on summarizer success (turnloop.go after non-empty summary). Gemini never reaches that branch with the stock ProviderSummarizer.

---

Live verification

Unit / package tests against current main (no Docker required for the prepare failure; full ProxyGeminiGenerateContent path exercised with in-memory fakes):

A. Format gate (ProviderSummarizer)

• Anthropic envelope → PrepareAnthropic succeeds → summarizer Proxy called.
• OpenAI envelope → same.
• Gemini envelope → error contains unsupported source format; summarizer Proxy call count = 0.

B. End-to-end Gemini SWITCH

• Pin: gemini-3.1-pro-preview (warm, prior usage).
• Fresh scorer decision: gemini-3.1-flash-lite-preview.
• Large Gemini contents (~10k estimated tokens) so EV is positive.
• Real proxy.NewProviderSummarizer wired; Google upstream is a capturing fake.

Observed:

• HeaderRouterModel = gemini-3.1-flash-lite-preview (SWITCH happened).
• Warn log with unsupported source format for Anthropic emit: 2.
• handover_fallback_to_full_history=true.
• Forwarded contents still contain original functionCall, functionResponse, and later user text.
• No [handover summary] tag in the forwarded body.
• Summarizer upstream call count = 0.

C. OpenAI contrast (same orchestrator)

• Analogous pin/SWITCH on OpenAI with a working fake summarizer → handover_fallback_to_full_history=false and summary tag present in the forwarded messages.

---

Impact

Dimension	Effect
Cost	Every Gemini SWITCH pays full prior contents input tokens at the new model rate. Anthropic/OpenAI SWITCH can collapse to summary + latest user. Long agent sessions amplify the gap.
Correctness today	Safe. Full-history fallback preserves tool pairing; no orphan-functionResponse 400 on this path.
Latency	Prepare fails locally (no summarizer RTT). Negligible vs the extra upstream input tokens on the switch turn.
Ops	Warn on every Gemini SWITCH (Handover summarizer failed; preserved full history instead).
Planner EV	Eviction cost is already priced against estimated full-prefix tokens, so EV is not “wrong”; the handover bonus that reduces realized switch-turn cost on other surfaces is simply never applied.

---

Required sibling when fixing (latent)

rewriteGeminiForHandover (internal/translate/handover.go) keeps the latest user turn verbatim. Anthropic’s sibling strips orphan tool_result blocks (and drops a tool-result-only user). Gemini does not strip orphan functionResponse parts.

Today that latent defect is unreachable on the Gemini SWITCH path because RewriteEnvelope never runs after a successful summary. As soon as summarizer ingest is fixed, a mid-tool SWITCH that successfully rewrites becomes:

[
  {"role":"model","parts":[{"text":"[handover summary] ..."}]},
  {"role":"user","parts":[{"functionResponse":{"name":"edit", ...}}]}
]

Google requires a functionResponse to follow a matching functionCall turn → 400 INVALID_ARGUMENT. Fix orphan stripping in the same change set as summarizer ingest (mirror stripAnthropicToolResultMsg / Anthropic RewriteForHandover behavior). Same-format PrepareGemini does not filter orphans (passthrough + tool schema sanitize only).

---

Related gaps (out of scope for this issue, same surface)

• maybeCompact is called from ProxyMessages and ProxyOpenAIChatCompletion only — not from ProxyGeminiGenerateContent. Separate under-wiring; do not block this fix on it.
• Cross-format Gemini→non-Google dispatch is explicitly deferred (ErrGeminiCrossFormatUnsupported → 501). This issue is about same-provider Google SWITCH still needing a summarizer that can read a Gemini envelope.

---

Suggested fix direction (not a patch)

1. Teach summarizer ingest to accept Gemini envelopes, e.g.:
   • add FormatGemini → Anthropic conversion in PrepareAnthropic / a dedicated builder used only by buildSummaryRequestBody, or
   • add a Gemini-native summarizer implementation.
2. In the same PR: strip orphan functionResponse parts from the latest user in rewriteGeminiForHandover (and drop the user turn if nothing remains), matching Anthropic.
3. Add regression tests:
   • ProviderSummarizer.Summarize succeeds (or at least prepares) on a Gemini envelope.
   • Gemini SWITCH e2e: handover_fallback_to_full_history=false and forwarded body is summary-bounded when summarizer returns text.
   • Gemini mid-tool rewrite: no orphan functionResponse after RewriteEnvelope + PrepareGemini.

Do not reintroduce silent TrimLastN on summarizer failure (handover AGENTS invariant).

---

Not already tracked

Checked open issues/PRs for Gemini handover / PrepareAnthropic / RewriteEnvelope / summarizer ingest. Closest open Gemini under-wiring items are #730 (force-model / pin eviction) and #731 (tool-call loop detection) — different call sites. Merged #637 added Gemini RewriteEnvelope happy-path unit tests but did not wire summarizer ingest; #203 fixed Anthropic/OpenAI orphan strip only.