Force-flush trace spans before the A2A response completes for substrate

[iplay88keys]

Description

Agent Substrate checkpoints an actor as soon as its A2A response completes. Spans still buffered by OpenTelemetry's BatchSpanProcessor at that point remain frozen until the actor resumes, or can be lost when the snapshot does not preserve process state. This makes the session's latest trace delayed or missing.

The flush must happen after HTTP instrumentation ends the inbound server span, but before the terminal response message triggers checkpointing. Flushing from the agent executor is too early because the server span is still open.

Changes

Go ADK

• Add telemetry.ForceFlush(ctx) to flush providers that support it.
• Wrap the instrumented A2A handler so flushing happens after the otelhttp server span ends and before ServeHTTP returns.
• Use a detached timeout because the request context may already be canceled.
• Skip health and agent-card requests.

Python ADK

• Add kagent.core.tracing.force_flush().
• Wrap the FastAPI ASGI stack and hold only the terminal response body or trailer message.
• Let the instrumented application return and end its server span, flush in a worker thread, then send the held terminal message.
• Skip health, thread-dump, and agent-card requests.

Both implementations default to a 3-second flush timeout, configurable with KAGENT_TRACE_FLUSH_TIMEOUT_MS. Unsupported tracer providers are a no-op, and exporter failures do not fail the A2A response. Streaming response chunks are unaffected; only response completion waits for the bounded flush.

Helm image helpers

• Include the optional image.name segment in the grafana-mcp and querydoc image helpers.
• Preserve existing two-segment image references when image.name is unset.

[Copilot]

Pull request overview

This PR ensures OpenTelemetry spans are force-flushed before an A2A response completes on Agent Substrate (to avoid buffered spans being checkpoint-frozen and exported only on the next resume), and updates Helm helper templates to optionally include an image.name segment when constructing image references.

Changes:

• Add Go telemetry.ForceFlush(ctx) and invoke it after the root invocation span ends in the Go A2A executor.
• Add Python kagent.core.tracing.force_flush() and invoke it at the end of the Python ADK executor via asyncio.to_thread to avoid blocking the event loop.
• Update grafana-mcp and querydoc Helm image helpers to include an optional image.name segment in the image path.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
python/packages/kagent-core/tests/test_tracing_configure.py	Adds unit tests for the new Python tracing force-flush helper behavior.
python/packages/kagent-core/src/kagent/core/tracing/_utils.py	Introduces force_flush() to flush buffered spans with a bounded timeout and swallow exporter errors.
python/packages/kagent-core/src/kagent/core/tracing/init.py	Exports force_flush from the tracing package public API.
python/packages/kagent-adk/src/kagent/adk/_agent_executor.py	Calls tracing force-flush in execute() cleanup using asyncio.to_thread.
helm/tools/querydoc/templates/_helpers.tpl	Extends image reference construction to include optional image.name.
helm/tools/grafana-mcp/templates/_helpers.tpl	Extends image reference construction to include optional image.name.
go/adk/pkg/telemetry/tracing.go	Adds ForceFlush helper that detaches from request cancellation and flushes spans with a 3s timeout.
go/adk/pkg/telemetry/tracing_test.go	Adds test ensuring buffered spans are exported on flush and that no-op providers don’t panic.
go/adk/pkg/a2a/executor.go	Flushes spans after ending the invocation span in deferred executor cleanup.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[krisztianfekete]

Can we gate this to substrate only?