Skip to content

Cost & telemetry parity: charged USD lost, cap-summary/failed-run usage unrecorded, elapsed_ms=0, tool DomainEvents gone #4467

Description

@senamakel

Part of the v0.58.7 → HEAD TinyAgents-migration audit (see parent issue). Cost-accounting and telemetry parity losses on the tinyagents path.

Cost accounting

  1. Backend-charged USD replaced by catalog estimates everywhere (src/openhuman/tinyagents/observability.rs:240-245): always estimate_cost_usd, never charged_amount_usd. Self-acknowledged follow-up; dashboards/footers now show estimates, not actual billing. Also record_provider_usage gets context_window: 0.
  2. Cap-hit summary usage under-accounted (ops/graph.rs:320-327): folds only input/output tokens — legacy also folded cached_input_tokens, priced the call, and fed cost::record_provider_usage. The summary call bypasses the harness so the bridge never sees it → cached tokens lost, $0 for that call in footer/transcript meta/cost dashboard. (Parent cap branch folds all four — asymmetry confirms unintended.)
  3. Failed unobserved runs record zero spend: record_unobserved_turn_usage (tinyagents/mod.rs:767) runs only on the success path; the Err arm returns first. A background/cron run burning N model calls before failing contributes nothing to wallet/cost surfaces. Legacy observer recorded per call regardless of outcome.

Telemetry

  1. ToolCallStarted projected with arguments: Null; ToolCallCompleted with output_chars: 0, elapsed_ms: 0 (observability.rs:495-551); ToolCallRecord.duration_ms always 0 (session/turn/core.rs:88); execute_openhuman_tool sets elapsed_ms: 0 on success (tinyagents/tools.rs:243). Legacy populated real args/durations/sizes. Args-aware display_label/display_detailhumanize_tool_name + None. The progress.rs variant docs still promise the full fields.
  2. DomainEvent::ToolExecutionStarted/Completed no longer published for session tool calls — SSE consumers lost per-tool telemetry.
  3. Child runs now emit unscoped ToolCallArgsDelta (observability.rs:357-379) — parent timeline may render child activity as its own (v0.58.7 dropped child arg fragments).
  4. Hallucinated (unknown) tool calls recorded as success: true: the crate's ReturnToolError path never runs after_tool (crate agent_loop/mod.rs:596-614), so the capture sink has no outcome and tool_records_from_conversation (core.rs:77-78) defaults to success; the cap digest also reports every tool [ok] (ops/graph.rs:554-582) despite outcome.tool_outcomes being available.
  5. ToolCallContext.iteration hardcoded 1 (middleware.rs:1316); cancelled web runs leave a non-terminal HarnessRunStatus in the journal/status store.

Fix plan

  1. Thread charged_amount_usd from provider responses through the observability bridge (restore legacy precedence: charged > estimate); pass real context_window.
  2. Route the cap-summary call through the same usage-recording path as harness calls (fold all four token fields + price it), matching the parent branch.
  3. Record usage in the Err arm of run_turn_via_tinyagents_shared before returning.
  4. Measure and populate elapsed_ms/output_chars/arguments in execute_openhuman_tool and the bridge projections; restore per-tool display labels or update the progress.rs docs to match reality.
  5. Re-publish DomainEvent::ToolExecutionStarted/Completed for session tool calls (from SharedToolAdapter or the capture middleware).
  6. Scope child ToolCallArgsDelta (add subagent scope field) or drop them as before.
  7. Default missing tool outcomes to success: false for unknown tools; derive cap-digest per-tool status from outcome.tool_outcomes.

Acceptance criteria

  • Footer/dashboard show backend-charged USD when available; failed cron runs show real spend; cap-summary call is priced with cached tokens.
  • Tool timeline shows real args, durations, output sizes; unknown-tool rows are marked failed.
  • SSE per-tool events restored.

Metadata

Metadata

Assignees

No one assigned

    Labels

    agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.bugpriority: mediumEdge case or race conditionsubtaskSubtask of a larger tracked effort.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions