Skip to content

Conversation

@eulervoid
Copy link

Description:
With the OpenAI API, completion_tokens is meant to be the total number of completions tokens, including the ones further specified in completion_tokens_details, like reasoning_tokens. The current behavior of the google provider in portkey is to return them separately, which leads to incorrect usage calculations down the line.

This PR tries to fix this by:

  • Calculating completion_tokens as the sum of candidatesTokenCount and thoughtsTokenCount to match the convention of the OpenAI API. I think this should probably also apply to audio_tokens, but i'm not as confident there, which is why i'm focusing on reasoning first with this PR.
  • Adding this update for both google and google-vertex-ai providers.

Tests Run/Test cases added:

  • No additional tests added
  • Unfortunately, i wasn't able to run all the tests due to missing credentials

Type of Change:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)

…sum of candidatesTokenCount and thoughtsTokenCount to match OpenAI API expectations
@narengogi
Copy link
Collaborator

valid @eulervoid reviewing

@narengogi
Copy link
Collaborator

with vertex, I just want to verify if the behaviour is same as gemini for anthropic models on vertex also

Copy link
Collaborator

@narengogi narengogi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, let me update this PR with the relevant changes for anthropic on vertex as well

@VisargD
Copy link
Collaborator

VisargD commented Dec 22, 2025

@narengogi - Should we wait for your updates?

@narengogi
Copy link
Collaborator

fixing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants