Skip to content

fix(anthropic): omit temperature instead of sending null when it is unset (#1794) - #1797

Open
Anai-Guo wants to merge 1 commit into
getzep:mainfrom
Anai-Guo:fix-anthropic-null-temperature
Open

fix(anthropic): omit temperature instead of sending null when it is unset (#1794)#1797
Anai-Guo wants to merge 1 commit into
getzep:mainfrom
Anai-Guo:fix-anthropic-null-temperature

Conversation

@Anai-Guo

Copy link
Copy Markdown

Fixes #1794.

Problem

mcp_server/src/config/schema.py defaults llm.temperature to None, and
mcp_server/src/services/factories.py passes it straight through to the core
LLMConfig — with the same comment repeated at all five provider branches:

# None is intentional for reasoning models; core LLMConfig stores it
# verbatim and downstream clients omit temperature when it is None.
temperature=config.temperature,  # type: ignore[arg-type]

OpenAIClient honors that contract (openai_client.py):

# Omit temperature entirely for reasoning models — don't even send None.
if not is_reasoning_model and temperature is not None:
    request_kwargs['temperature'] = temperature

AnthropicClient did not — it forwarded self.temperature to
messages.create() unconditionally. The Anthropic SDK serializes an explicit
None rather than dropping the key, so the request body carries
"temperature": null and the API rejects it:

400 - {'type': 'error', 'error': {'type': 'invalid_request_error',
       'message': 'temperature: Input should be a valid number'}}

Which I confirmed at the wire level against a mocked httpx transport
(anthropic 0.120.2):

call key on the wire value
messages.create(..., temperature=None) present null
messages.create(...) (omitted) absent
messages.create(..., temperature=0.0) present 0.0

Under the MCP server this is invisible to callers: the episode queue catches the
exception, logs it and marks the task done, so every add_memory still reports
success while nothing lands.

Fix

Build the messages.create() kwargs and add temperature only when it is not
None, mirroring OpenAIClient._create_completion. The guard is is not None,
not truthiness, so an explicit temperature=0.0 is still sent.

Verification

Reproduction against the real AnthropicClient with a mocked
messages.create, before → after:

config before after
temperature unset (None) sent as None omitted
temperature=0.0 sent as 0.0 sent as 0.0
temperature=0.7 sent as 0.7 sent as 0.7

Tests (tests/llm_client/test_anthropic_client.py, new
TestAnthropicClientTemperature) cover all three rows.

  • Full file: 11 passed before, 14 passed after.
  • Red/green: reverting only anthropic_client.py while keeping the new tests
    fails exactly test_temperature_omitted_when_unset (13 passed, 1 failed).
    The two guard cases stay green on both sides, which is what they are for.
  • pytest tests/llm_client tests/test_text_utils.py tests/utils: 297 passed,
    2 skipped
    (the skips are the _int tests, no ANTHROPIC_API_KEY).
  • ruff==0.14.11 (the version pinned in .github/workflows/lint.yml)
    check and format --check clean on both files.

Deliberately not in scope

The issue reports two things; this PR fixes only the first.

  • OpenAIGenericClient, GeminiClient, GroqClient also pass
    temperature=self.temperature through unconditionally, so the factory's
    "downstream clients omit temperature when it is None" comment is not true
    for them either. Whether that is actually a bug depends on how each of those
    APIs treats an explicit null, which I have not verified the way I verified
    Anthropic's — so I left them alone rather than guess. Happy to follow up if
    you'd like them covered.
  • Surfacing per-episode queue failures (the issue's second half — a queue
    that turns hard failures into log lines while the tool call reports success).
    That is an MCP server API-shape decision, not a bug fix.

🤖 Generated with Claude Code

…nset (getzep#1794)

The MCP server's config schema defaults llm.temperature to None, and the
client factory stores it verbatim on the core LLMConfig, noting that
"downstream clients omit temperature when it is None". The OpenAI client
does exactly that, but AnthropicClient forwarded self.temperature to
messages.create() unconditionally.

The Anthropic SDK serializes an explicit None rather than dropping the
key, so the request body carries "temperature": null and the API answers
400 invalid_request_error ("temperature: Input should be a valid number").

Build the create() kwargs and add temperature only when it is not None,
mirroring OpenAIClient._create_completion. An explicit 0.0 is still sent.

@linhongyu510 linhongyu510 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at 6e51b5c. The change is correctly limited to request construction: temperature is omitted only for None, while explicit 0.0 and 0.7 remain present, with no changes to retries, token accounting, tool parsing, or exception mapping. Independent validation: the full Anthropic client module passes 14 tests; tests/llm_client tests/test_text_utils.py tests/utils passes 319 tests with 2 expected integration skips; Ruff check/format and git diff --check pass. I also performed a red/green check by reverting only anthropic_client.py while retaining the new test: test_temperature_omitted_when_unset fails exactly because the baseline still forwards the key, then all three temperature tests pass again on this HEAD. The separate queue-failure observability concern is appropriately left out of this minimal fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AnthropicClient sends temperature: null when temperature is unset (400 from API) — and the MCP queue then silently drops every episode

2 participants