Skip to content

fix(llm): keep Gemini thought signatures on replayed tool calls - #1044

Open
ZhuLinsen wants to merge 1 commit into
modelscope:mainfrom
ZhuLinsen:fix/gemini-thought-signature-tool-calls
Open

ZhuLinsen wants to merge 1 commit into
modelscope:mainfrom
ZhuLinsen:fix/gemini-thought-signature-tool-calls

Conversation

@ZhuLinsen

Copy link
Copy Markdown

Change Summary

Gemini 3 tool loops stop after the first tool result with 400 Function call is missing a thought_signature. On Gemini's OpenAI-compatible endpoint the signature comes back on each tool call as extra_content.google.thought_signature. Google requires it to be sent back, and ms-agent dropped it.

This PR keeps extra_content on ToolCall:

  • Both OpenAI-compatible clients, transport/openai_compat.py and the legacy openai_llm.py, read it in the streaming and non-streaming parsers.
  • Stream merging keeps it whichever chunk carries it.
  • Message.to_dict_clean puts it back on the wire tool call.

The field is written only when the provider returned it, so requests to other OpenAI-compatible providers are unchanged. Session logs already keep it, because ToolCall is a plain dict.

Known limits, left out to keep this focused:

  • If a session with Gemini tool calls is replayed against a different provider, the old calls still carry extra_content.
  • The stream mergers only extend the last tool call. A signature-only delta for an earlier call that arrives after a later call has started would still be misplaced. This limit already applies to arguments.
  • ResponseAdapter.to_message does not carry the field. It has no caller in the repo, and the agent loop does not use it.

The reasoning_content point in the issue comments is a separate change and is not included here.

Related issue number

fix #1040

Checklist

  • The pull request title is a good summary of the changes - it will be used in the changelog
  • Unit tests for the changes exist
  • Run pre-commit install and pre-commit run --all-files before git commit, and passed lint check.
  • Documentation reflects the changes where applicable

Tests

New tests/llm/test_tool_call_extra_content.py uses real openai ChatCompletion and ChatCompletionChunk objects with extra_content. It covers both clients for:

  • a non-streaming round trip;
  • streaming, with the signature arriving on different chunks of two parallel calls;
  • providers that send no field, which must get no field back;
  • the assistant turn sent in the next request.

Without the fix, 3 of the 4 tests fail with KeyError: 'extra_content'. The test that checks no field is added passes either way.

pytest tests/llm/test_tool_call_extra_content.py tests/llm/test_toolcall_empty_content.py -q
11 passed

tests/llm/test_message_text.py, test_continuation_guard.py and test_thinking_fallback.py also pass. pre-commit run --files was run on the changed files. yapf also wanted to reformat three unrelated existing lines in the touched files, and I left those out of the diff. I tested against the parsing code only, not against the live Gemini API.

Gemini 3 returns each tool call's thought signature as
extra_content.google.thought_signature on its OpenAI-compatible endpoint and
answers 400 when the next request replays the call without it. Keep the field
through both OpenAI-compatible parsers and stream merging, and send it back in
Message.to_dict_clean only when the provider returned it.

fix modelscope#1040

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[llm] Gemini 3 tool loop fails with 400: thought signature is not sent back

1 participant