Skip to content

[ApexAgents] Coerce stringified MCP tool arguments and surface validation errors - #2868

Merged
arti4nvj merged 3 commits into
NVIDIA-NeMo:arti/apexagentsfrom
Kh4L:spanev/apex-glm-tool-arg-coercion
Aug 31, 2026
Merged

[ApexAgents] Coerce stringified MCP tool arguments and surface validation errors#2868
arti4nvj merged 3 commits into
NVIDIA-NeMo:arti/apexagentsfrom
Kh4L:spanev/apex-glm-tool-arg-coercion

Conversation

@Kh4L

@Kh4L Kh4L commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Problem

GLM 5.2 rollouts fail 100% of tool calls to Archipelago MCP tools whose schema wraps parameters in a bare-$ref property with no inline type (e.g. {"properties": {"request": {"$ref": "#/$defs/DynamicModel"}}}). vLLM's glm47 tool-call parser reconstructs argument types from properties[key].type; with no inline type the object value arrives as a JSON-encoded string ({"request": "{\"code\": ...}"}), Stirrup's pydantic validation rejects it with the bare Tool arguments are not valid, and the model retries the same shape indefinitely. Observed in the kimi-vs-glm52 trajectory comparison: 12/12 code_code_exec failures, 3/3 mail_search_mail, 113 trajectory items and 3 context compactions vs 13 items for Kimi-K3 on the same task. Flat inline-typed schemas (todo_write, mail_read_mail) are unaffected.

The typing bug is serving-side (same family as vLLM PR #41801 for deepseek_v4), so the model cannot self-correct no matter how the error is phrased — the harness has to handle it.

Changes

Repair the symptom (coercion safety net):

  • stirrup_runtime.py: add coerce_tool_arguments, a validation-gated repair that (a) unwraps top-level string values that JSON-decode to an object/array where the field annotation expects structured data, and (b) wraps flat arguments into the model's single required nested-model field. A candidate is accepted only if it validates, and dict payloads may only use keys the nested model knows (MCP-generated models default to extra="ignore", so validation alone would silently drop unknown keys and run the tool with defaults).
  • Patch Agent.run_tool via install_tool_argument_coercion, installed inside run_stirrup_rollout (the code must be self-contained in this file — it is the only one staged into the sandbox). Coercion uses model_copy so trajectory history keeps the model's raw emission. Finish tools are exempt: Agent.step() re-validates the original tool_call outside any try/except (stirrup 0.1.12 agent.py:1261).
  • When coercion cannot repair the args, surface full pydantic error detail plus an args preview instead of the bare message (same approach as stirrup_agent/nemo_client.py).

Remove the trigger (type annotation at the client boundary):

  • annotate_schema_ref_types: copies only the resolved definition's type onto typeless $ref sites — $ref and $defs stay intact. A type sibling equal to the definition's type is a conjunctive no-op under JSON Schema 2020-12 (and ignored under draft-07), so schema semantics are provably unchanged, while the parser now finds properties[key].type and stops string-encoding. Full inlining was considered and rejected: merging sibling keys alters 2020-12 conjunctive constraint semantics (e.g. a site maxLength overriding a stricter definition maxLength). Data-position values (const/enum/default/examples) are never annotated, and definition names are RFC 6901-escaped during resolution.
  • install_tool_schema_type_annotation wraps stirrup's to_openai_tools and rebinds it in stirrup.clients.utils and the from-import namespaces (chat_completions_client, litellm_client) — the client calls the copy in its own namespace. inspect_tool output shows the same annotated shape the wire carries.
  • The two layers compose: type annotation fixes type reconstruction for schema-aware parsers; coercion still repairs parsers that string-encode regardless.

Testing

  • pytest responses_api_agents/apex_agent/tests/ — 62 passed (41 pre-existing, no regressions).
  • ruff check / ruff format --check clean; module-level imports of stirrup_runtime.py verified stdlib-only (bare python3 import without stirrup installed).
  • Verified end-to-end against the real json_schema_to_pydantic-generated model that MCPToolProvider builds: all three observed GLM shapes repair; garbage and already-valid args are left alone; the patched to_openai_tools in the ChatCompletionsClient namespace emits {"$ref": ..., "type": "object"} at the property site with $defs preserved; raw-vs-annotated schemas agree on ~10k adversarial + fuzzed instances under jsonschema Draft 2020-12 and Draft-07, including the conflicting-maxLength conjunction case.

…tion errors

Signed-off-by: Serge Panev <spanev@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 29, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: Serge Panev <spanev@nvidia.com>
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Aug 31, 2026
resolved = resolve(definition_tables[ref], expanding | {ref})
siblings = {key: resolve(value, expanding) for key, value in node.items() if key != "$ref"}
if isinstance(resolved, dict):
return {**resolved, **siblings}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One concern here is that sibling constraints are applied in addition to the reference schema rather than overwriting it. So if def has maxlen=1 and $ref site as maxlen=3, the merge would set maxlen=3 overwriting and accepting the value that the og schema rejects

could we preserve the $ref and $defs and copy the resolved type onto the reference site instead?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, fixed in the latest commit. Implemented what you suggested: keep $ref/$defs, copy only the resolved type onto the ref site.

@github-actions github-actions Bot removed the sla:triage-overdue Review assignment is over the one-business-day SLA label Aug 31, 2026
Signed-off-by: Serge Panev <spanev@nvidia.com>
@arti4nvj
arti4nvj merged commit 8760c53 into NVIDIA-NeMo:arti/apexagents Aug 31, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants