Display TensorRT-LLM AgentX server metrics

[cquil11]

Summary

• add a TensorRT-LLM server-metrics adapter for Dynamo disaggregated roles
• expose TensorRT-LLM token, cache, queue, and KV metrics in AgentX charts and aggregates
• preserve separate prefill and decode metric sources

Follows SemiAnalysisAI/InferenceX#2625 and NVIDIA/TensorRT-LLM#12545.

Validation

• 24 focused Vitest tests passed
• oxlint passed
• oxfmt check passed
• TypeScript typecheck passed

---

Note

Medium Risk
Changes versioned ETL that shapes chart_series and aggregate_stats for production benchmarks; incorrect mapping could misreport throughput or cache metrics until backfill, but scope is an additive framework branch with focused tests.

Overview
Adds TensorRT-LLM as a first-class source for AgentX server-metrics charts and aggregate stats, alongside existing vLLM and SGLang paths.

Charts (compute-chart-series) — CHART_SERIES_VERSION is bumped to 17. Native trtllm_* metrics now drive KV utilization, host KV, queue depth, decode throughput, prefix-cache hit rate, and prompt-token breakdowns. When TRT-LLM lacks a prompt-token counter, prefill TPS is derived from trtllm_prefill_batch_tokens histogram sum per timeslice (new sumRate handling) plus cached-token rates. Disaggregated Dynamo runs get separate prefill/decode metricSources via a new trtllm server-metrics adapter that maps dynamo_component / disaggregation_mode labels.

Aggregates — The same TRT-LLM metric keys are included in stream-parse extraction for oversized blobs (compute-aggregate-stats, agentic-aggregates), with fallbacks to trtllm_kv_cache_hit_rate when hits/queries counters are missing.

Tests cover TRT-LLM chart extraction and aggregate sampling; the API route catalog SHA for agentic-aggregates is updated.

^{Reviewed by Cursor Bugbot for commit 10cdc42. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
inferencemax-app	Ready	Preview	Aug 28, 2026 3:57pm

[cquil11]

Closing because the project scope is now limited to porting active, nondeprecated AMD multi-node configurations to srt-slurm. This PR is outside that scope.

[cquil11]

Reopening: this PR predates the recent out-of-scope configuration-porting detour and was closed by mistake during an overly broad cleanup. Its prior state is being restored.

[cursor]

TRT-LLM adapter drops Dynamo router role

Medium Severity

The new trtllm adapter is listed before the Dynamo adapter and matches any framework whose name contains trt, including canonical dynamo-trt. Its role mapping handles prefill/backend/decode/aggregated but not Dynamo frontend/router, so those sources are stored as unknown instead of router. Per-source pickers for Dynamo TRT-LLM disagg runs therefore mislabel the frontend endpoint.

Additional Locations (1)

• packages/db/src/etl/server-metrics-adapters.ts#L79-L87

 

^{Reviewed by Cursor Bugbot for commit 4e703c0. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1e48786. Configure here.}

[cursor]

TRT-LLM adapter splits worker sources

High Severity

The new trtllm adapter matches any framework containing trt, including dynamo-trt, and keys each source by role, endpointUrl, workerId, dpRank, and engine. That undoes worker-level identity, so one Dynamo worker with multiple ranks or inconsistently labeled series appears as many endpoints with undercounted per-source throughput.

Additional Locations (1)

• packages/db/src/etl/compute-chart-series.ts#L1206-L1223

 

^{Reviewed by Cursor Bugbot for commit 1e48786. Configure here.}

[cursor]

Aggregate version not bumped

Medium Severity

TensorRT-LLM keys were added to extractServerMetricSamples, but STATS_VERSION remains 8. The read path and backfill only recompute when the stored version differs, so existing TensorRT-LLM rows keep empty kvCacheUtil and prefixCacheHitRate. Chart extraction correctly bumped CHART_SERIES_VERSION.

Additional Locations (1)

• packages/db/src/etl/compute-aggregate-stats.ts#L195-L201

 

^{Reviewed by Cursor Bugbot for commit 1e48786. Configure here.}