Skip to content

adk web: 'System Instruction Performance Analysis' false-positives in multi-agent sessions #462

Description

@vinismarques

Summary

The dev UI's System Instruction Performance Analysis warning ("System instructions were modified between consecutive turns in this session. This change breaks context cache alignment, resulting in a cache miss...") fires whenever a session contains more than one agent, even though no system instruction was actually modified.

updateSystemInstructionFlags() compares system instructions across all LLM spans in a session, sorted by time, without partitioning by agent. In any multi-agent / sub-agent flow, crossing an agent boundary makes the system instruction legitimately differ, which trips the warning. The diff dialog then shows two different agents' prompts, which is confusing.

Where

src/app/components/chat/chat.component.tsupdateSystemInstructionFlags():

const llmSpans = flatSpans
  .filter(s => {
    const isGCSpan = s.attrOperationName === OPERATION_GENERATE_CONTENT;
    const isLegacySpan = s.name === 'call_llm';
    return (isGCSpan || isLegacySpan) && s.io?.inputs !== undefined;
  })
  .sort((a, b) => (a.start_time || 0) - (b.start_time || 0));

// Compare consecutive LLM turns
for (let i = 1; i < llmSpans.length; i++) {
  const currentSpan = llmSpans[i];
  const precedingSpan = llmSpans[i - 1];

  const currentSys = extractSystemInstruction(currentSpan.io?.inputs);
  const precedingSys = extractSystemInstruction(precedingSpan.io?.inputs);

  if (currentSys && precedingSys && currentSys !== precedingSys) {
    // ...flag event.systemInstructionChanged = true
  }
}

llmSpans spans the entire session across every agent, so consecutive spans from different agents get compared directly. There is no grouping by agent. The agent is the correct boundary: each agent has its own system instruction, and therefore its own cache context. "System instruction modified between turns" is only meaningful within a single agent's own sequence of calls. Two distinct agents never shared a cache to align in the first place, so comparing them is a category error — independent of whether they happen to use the same model or provider.

Steps to reproduce

  1. Run a root agent that delegates to a sub-agent (e.g. via transfer_to_agent / AgentTool), each with its own static instruction.
  2. In adk web, send one message that causes the root agent to call the sub-agent and then respond.
  3. The session's LLM calls end up ordered e.g. root -> sub -> sub -> ... -> root.
  4. The warning appears, and the diff shows the root agent's prompt vs the sub-agent's prompt.

Neither system instruction was edited; the only "change" is the agent boundary.

Expected

  • No warning when the system instruction is stable within a given agent's own consecutive turns.
  • The comparison should be grouped per agent (e.g. per event author / agent invocation), not across the whole session timeline.

Proposed fix

In updateSystemInstructionFlags(), group llmSpans by agent (e.g. the event author / agent invocation, via eventData.get(span.attrEventId)) before the consecutive comparison, so systemInstructionChanged only reflects a real change within a single agent's own turns (e.g. a dynamic / state-templated instruction that genuinely changes between that agent's calls).

Environment

  • adk-web dev UI (bundled with adk web)
  • Multi-agent setup (root agent + sub-agents)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions