-
-
Notifications
You must be signed in to change notification settings - Fork 385
fix: apply matched provider group billing multiplier #1327
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,7 +4,11 @@ import { PROVIDER_GROUP } from "@/lib/constants/provider.constants"; | |
| import { logger } from "@/lib/logger"; | ||
| import { RateLimitService } from "@/lib/rate-limit"; | ||
| import { SessionManager } from "@/lib/session-manager"; | ||
| import { parseProviderGroups, resolveProviderGroupsWithDefault } from "@/lib/utils/provider-group"; | ||
| import { | ||
| parseProviderGroups, | ||
| resolveBillingProviderGroups, | ||
| resolveProviderGroupsWithDefault, | ||
| } from "@/lib/utils/provider-group"; | ||
| import { isProviderActiveNow } from "@/lib/utils/provider-schedule"; | ||
| import { resolveSystemTimezone } from "@/lib/utils/timezone"; | ||
| import { isVendorTypeCircuitOpen } from "@/lib/vendor-type-circuit-breaker"; | ||
|
|
@@ -63,6 +67,46 @@ function checkProviderGroupMatch(providerGroupTag: string | null, userGroups: st | |
| return providerTags.some((tag) => groups.includes(tag)); | ||
| } | ||
|
|
||
| async function resolveGroupCostMultiplierForProvider(session: ProxySession): Promise<void> { | ||
| const effectiveGroup = getEffectiveProviderGroup(session); | ||
| const provider = session.provider; | ||
|
|
||
| if (!effectiveGroup || !provider) { | ||
| session.setGroupCostMultiplier(1.0); | ||
| return; | ||
| } | ||
|
|
||
| const billingGroups = resolveBillingProviderGroups(provider.groupTag, effectiveGroup); | ||
| if (billingGroups.length === 0) { | ||
| logger.warn( | ||
| "[ProviderResolver] Selected provider has no billing group intersection, falling back to 1.0", | ||
| { | ||
| providerId: provider.id, | ||
| providerName: provider.name, | ||
| providerGroups: provider.groupTag, | ||
| effectiveGroup, | ||
| } | ||
| ); | ||
| session.setGroupCostMultiplier(1.0); | ||
| return; | ||
| } | ||
|
|
||
| const billingGroup = billingGroups.join(","); | ||
|
|
||
| try { | ||
| const multiplier = await getGroupCostMultiplier(billingGroup); | ||
|
Comment on lines
+94
to
+97
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a provider carries multiple group tags that all appear in the user's declared groups (e.g. provider This is consistent with Prompt To Fix With AIThis is a comment left during a code review.
Path: src/app/v1/_lib/proxy/provider-selector.ts
Line: 94-97
Comment:
**Multi-group intersection joined before lookup**
When a provider carries multiple group tags that all appear in the user's declared groups (e.g. provider `"group-a,group-b"`, user `"group-a,group-b"`), `resolveBillingProviderGroups` returns `["group-a","group-b"]` and `billingGroups.join(",")` produces `"group-a,group-b"`. `getGroupCostMultiplier` then applies first-match on the parsed list, so `group-a`'s multiplier wins regardless of which group carries the higher rate.
This is consistent with `getGroupCostMultiplier`'s documented "first declared group wins" contract, so it isn't a new regression. However, neither the integration tests in this PR nor the unit tests for `resolveBillingProviderGroups` exercise the end-to-end billing path when the intersection contains more than one element. A case where both groups match the provider but have different multipliers would silently use user-declaration order as the tiebreaker without any test cover.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||
| session.setGroupCostMultiplier(multiplier); | ||
| } catch (error) { | ||
| logger.warn("[ProviderResolver] Failed to resolve group cost multiplier, falling back to 1.0", { | ||
| billingGroup, | ||
| effectiveGroup, | ||
| providerId: provider.id, | ||
| error: error instanceof Error ? error.message : String(error), | ||
| }); | ||
| session.setGroupCostMultiplier(1.0); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * 检查供应商是否支持指定模型(用于调度器匹配) | ||
| * | ||
|
|
@@ -186,26 +230,6 @@ export class ProxyProviderResolver { | |
| session.setLastSelectionContext(context); // 保存用于后续记录 | ||
| } | ||
|
|
||
| // === Resolve group cost multiplier === | ||
| // Fail soft: if the lookup throws (Redis/DB hiccup), fall back to 1.0 so | ||
| // request handling proceeds without billing disruption. | ||
| const effectiveGroup = getEffectiveProviderGroup(session); | ||
| if (effectiveGroup) { | ||
| try { | ||
| const multiplier = await getGroupCostMultiplier(effectiveGroup); | ||
| session.setGroupCostMultiplier(multiplier); | ||
| } catch (error) { | ||
| logger.warn( | ||
| "[ProviderResolver] Failed to resolve group cost multiplier, falling back to 1.0", | ||
| { | ||
| effectiveGroup, | ||
| error: error instanceof Error ? error.message : String(error), | ||
| } | ||
| ); | ||
| session.setGroupCostMultiplier(1.0); | ||
| } | ||
| } | ||
|
|
||
| // === 故障转移循环 === | ||
| let attemptCount = 0; | ||
| while (true) { | ||
|
|
@@ -341,11 +365,13 @@ export class ProxyProviderResolver { | |
| // 修复:延迟到 forwarder 请求成功后统一更新(见 forwarder.ts:75-80) | ||
| // void SessionManager.updateSessionProvider(...); // ❌ 已移除 | ||
|
|
||
| await resolveGroupCostMultiplierForProvider(session); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When the provider chosen by Useful? React with 👍 / 👎. |
||
| return null; // 成功 | ||
| } | ||
|
|
||
| // sessionId 为空的情况(理论上不应该发生) | ||
| logger.warn("ProviderSelector: sessionId is null, skipping concurrent check"); | ||
| await resolveGroupCostMultiplierForProvider(session); | ||
| return null; | ||
| } | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Issue Analysis:
Currently,
resolveGroupCostMultiplierForProvideris a private, module-scoped function that is only called withinProxyProviderResolver.ensure.However, when a request fails with an upstream error (such as a 5xx or timeout) during the actual forwarding phase in
forwarder.ts, a cross-provider retry/fallback is typically triggered. In this scenario,forwarder.tscallspickRandomProviderWithExclusionto select a new fallback provider and updates the session's provider.Since
forwarder.tscannot access the privateresolveGroupCostMultiplierForProviderfunction, the billing multiplier will not be re-resolved for the newly selected fallback provider. As a result, the final billing will incorrectly use the multiplier of the initial (failed) provider, leading to billing discrepancies (overbilling or underbilling) in multi-group environments.Recommendation:
Export
resolveGroupCostMultiplierForProviderso thatforwarder.ts(or any external retry mechanism) can import and call it to update the group cost multiplier whenever the provider is switched during fallback.