Update gemini response transform: calculate completion_tokens as the sum of candidatesTokenCount and thoughtsTokenCount to match OpenAI API expectations #1462
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
With the OpenAI API,
completion_tokensis meant to be the total number of completions tokens, including the ones further specified incompletion_tokens_details, likereasoning_tokens. The current behavior of thegoogleprovider in portkey is to return them separately, which leads to incorrect usage calculations down the line.This PR tries to fix this by:
completion_tokensas the sum ofcandidatesTokenCountandthoughtsTokenCountto match the convention of the OpenAI API. I think this should probably also apply toaudio_tokens, but i'm not as confident there, which is why i'm focusing on reasoning first with this PR.googleandgoogle-vertex-aiproviders.Tests Run/Test cases added:
Type of Change: