Skip to content

fix(openai): accumulate streamed refusal and reasoning_content - #3797

Open
CodeWithMoin wants to merge 1 commit into
Arize-ai:mainfrom
CodeWithMoin:fix/openai-stream-accumulate-refusal-reasoning
Open

CodeWithMoin wants to merge 1 commit into
Arize-ai:mainfrom
CodeWithMoin:fix/openai-stream-accumulate-refusal-reasoning

Conversation

@CodeWithMoin

Copy link
Copy Markdown

Fixes #3785

What changed

On a streamed chat completion, refusal and reasoning_content are now accumulated across chunks, same as content. One change in _response_accumulator.py: both are declared as _StringAccumulator() in the message schema.

Why

Only content, function_call and tool_calls were declared there. Any other string field on the delta went through the new-entry path on the first chunk and the replacement path on every chunk after, so output.value kept only the last fragment. refusal is a first-party delta field, and reasoning_content is what DeepSeek, vLLM and Qwen3 stream through a base_url override. So a streamed trace showed a plausible but wrong value, and didn't match the same call with stream=False.

I kept it to these two fields rather than accumulating every undeclared string. Scalar fields like role can repeat across chunks and would get concatenated.

Trace output

output.value message for the same four streamed chunks (two reasoning_content, two content):

before: {"role": "assistant", "reasoning_content": "is 4.",          "content": "The answer is 4."}
after:  {"role": "assistant", "reasoning_content": "2 plus 2 is 4.", "content": "The answer is 4."}

Tests

New tests/openinference/instrumentation/openai/test_response_accumulator.py:

  • content, refusal and reasoning_content streamed over three chunks each come out whole, and role stays "assistant"
  • reasoning and content streamed in the same response accumulate independently

On main the refusal and reasoning_content cases fail and content passes. With the fix all 4 pass.

pytest tests   # 488 passed, 1 failed

The one failure is test_tool_calls. It fails the same way on main in a full run and passes alone, so it looks like test-order state and isn't related. mypy is clean on the changed files.

Only content, function_call and tool_calls were declared as accumulated
fields on a streamed chat completion message. refusal and
reasoning_content fell through to the new-entry/replacement path, so each
chunk overwrote the last and output.value kept only the final fragment.

Declare both as string accumulators, like content.

Fixes Arize-ai#3785
@github-actions

github-actions Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@CodeWithMoin

Copy link
Copy Markdown
Author

recheck

@CodeWithMoin

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Sep 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[bug] Streamed chat-completion delta fields outside the accumulator schema are replaced, not concatenated (refusal, reasoning_content)

1 participant