Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 47 additions & 21 deletions src/app/v1/_lib/proxy/provider-selector.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ import { PROVIDER_GROUP } from "@/lib/constants/provider.constants";
import { logger } from "@/lib/logger";
import { RateLimitService } from "@/lib/rate-limit";
import { SessionManager } from "@/lib/session-manager";
import { parseProviderGroups, resolveProviderGroupsWithDefault } from "@/lib/utils/provider-group";
import {
parseProviderGroups,
resolveBillingProviderGroups,
resolveProviderGroupsWithDefault,
} from "@/lib/utils/provider-group";
import { isProviderActiveNow } from "@/lib/utils/provider-schedule";
import { resolveSystemTimezone } from "@/lib/utils/timezone";
import { isVendorTypeCircuitOpen } from "@/lib/vendor-type-circuit-breaker";
Expand Down Expand Up @@ -63,6 +67,46 @@ function checkProviderGroupMatch(providerGroupTag: string | null, userGroups: st
return providerTags.some((tag) => groups.includes(tag));
}

async function resolveGroupCostMultiplierForProvider(session: ProxySession): Promise<void> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Issue Analysis:
Currently, resolveGroupCostMultiplierForProvider is a private, module-scoped function that is only called within ProxyProviderResolver.ensure.

However, when a request fails with an upstream error (such as a 5xx or timeout) during the actual forwarding phase in forwarder.ts, a cross-provider retry/fallback is typically triggered. In this scenario, forwarder.ts calls pickRandomProviderWithExclusion to select a new fallback provider and updates the session's provider.

Since forwarder.ts cannot access the private resolveGroupCostMultiplierForProvider function, the billing multiplier will not be re-resolved for the newly selected fallback provider. As a result, the final billing will incorrectly use the multiplier of the initial (failed) provider, leading to billing discrepancies (overbilling or underbilling) in multi-group environments.

Recommendation:
Export resolveGroupCostMultiplierForProvider so that forwarder.ts (or any external retry mechanism) can import and call it to update the group cost multiplier whenever the provider is switched during fallback.

Suggested change
async function resolveGroupCostMultiplierForProvider(session: ProxySession): Promise<void> {
export async function resolveGroupCostMultiplierForProvider(session: ProxySession): Promise<void> {

const effectiveGroup = getEffectiveProviderGroup(session);
const provider = session.provider;

if (!effectiveGroup || !provider) {
session.setGroupCostMultiplier(1.0);
return;
}

const billingGroups = resolveBillingProviderGroups(provider.groupTag, effectiveGroup);
if (billingGroups.length === 0) {
logger.warn(
"[ProviderResolver] Selected provider has no billing group intersection, falling back to 1.0",
{
providerId: provider.id,
providerName: provider.name,
providerGroups: provider.groupTag,
effectiveGroup,
}
);
session.setGroupCostMultiplier(1.0);
return;
}

const billingGroup = billingGroups.join(",");

try {
const multiplier = await getGroupCostMultiplier(billingGroup);
Comment on lines +94 to +97

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Multi-group intersection joined before lookup

When a provider carries multiple group tags that all appear in the user's declared groups (e.g. provider "group-a,group-b", user "group-a,group-b"), resolveBillingProviderGroups returns ["group-a","group-b"] and billingGroups.join(",") produces "group-a,group-b". getGroupCostMultiplier then applies first-match on the parsed list, so group-a's multiplier wins regardless of which group carries the higher rate.

This is consistent with getGroupCostMultiplier's documented "first declared group wins" contract, so it isn't a new regression. However, neither the integration tests in this PR nor the unit tests for resolveBillingProviderGroups exercise the end-to-end billing path when the intersection contains more than one element. A case where both groups match the provider but have different multipliers would silently use user-declaration order as the tiebreaker without any test cover.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/app/v1/_lib/proxy/provider-selector.ts
Line: 94-97

Comment:
**Multi-group intersection joined before lookup**

When a provider carries multiple group tags that all appear in the user's declared groups (e.g. provider `"group-a,group-b"`, user `"group-a,group-b"`), `resolveBillingProviderGroups` returns `["group-a","group-b"]` and `billingGroups.join(",")` produces `"group-a,group-b"`. `getGroupCostMultiplier` then applies first-match on the parsed list, so `group-a`'s multiplier wins regardless of which group carries the higher rate.

This is consistent with `getGroupCostMultiplier`'s documented "first declared group wins" contract, so it isn't a new regression. However, neither the integration tests in this PR nor the unit tests for `resolveBillingProviderGroups` exercise the end-to-end billing path when the intersection contains more than one element. A case where both groups match the provider but have different multipliers would silently use user-declaration order as the tiebreaker without any test cover.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

session.setGroupCostMultiplier(multiplier);
} catch (error) {
logger.warn("[ProviderResolver] Failed to resolve group cost multiplier, falling back to 1.0", {
billingGroup,
effectiveGroup,
providerId: provider.id,
error: error instanceof Error ? error.message : String(error),
});
session.setGroupCostMultiplier(1.0);
}
}

/**
* 检查供应商是否支持指定模型(用于调度器匹配)
*
Expand Down Expand Up @@ -186,26 +230,6 @@ export class ProxyProviderResolver {
session.setLastSelectionContext(context); // 保存用于后续记录
}

// === Resolve group cost multiplier ===
// Fail soft: if the lookup throws (Redis/DB hiccup), fall back to 1.0 so
// request handling proceeds without billing disruption.
const effectiveGroup = getEffectiveProviderGroup(session);
if (effectiveGroup) {
try {
const multiplier = await getGroupCostMultiplier(effectiveGroup);
session.setGroupCostMultiplier(multiplier);
} catch (error) {
logger.warn(
"[ProviderResolver] Failed to resolve group cost multiplier, falling back to 1.0",
{
effectiveGroup,
error: error instanceof Error ? error.message : String(error),
}
);
session.setGroupCostMultiplier(1.0);
}
}

// === 故障转移循环 ===
let attemptCount = 0;
while (true) {
Expand Down Expand Up @@ -341,11 +365,13 @@ export class ProxyProviderResolver {
// 修复:延迟到 forwarder 请求成功后统一更新(见 forwarder.ts:75-80)
// void SessionManager.updateSessionProvider(...); // ❌ 已移除

await resolveGroupCostMultiplierForProvider(session);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Recompute group multiplier after provider switches

When the provider chosen by ensure later fails upstream, ProxyForwarder can replace it with an alternative provider (src/app/v1/_lib/proxy/forwarder.ts:2341-2358) or a streaming hedge winner (src/app/v1/_lib/proxy/forwarder.ts:4533-4541), but those paths do not recalculate the new provider-specific group multiplier. Because this line now resolves billing from the provider selected before forwarding, a request that starts on a provider matching group-a and finishes on a fallback provider matching group-b is billed with the stale group-a multiplier. Please recalculate the multiplier whenever the final provider changes, or defer this resolution until the final provider is known.

Useful? React with 👍 / 👎.

return null; // 成功
}

// sessionId 为空的情况(理论上不应该发生)
logger.warn("ProviderSelector: sessionId is null, skipping concurrent check");
await resolveGroupCostMultiplierForProvider(session);
return null;
}

Expand Down
30 changes: 30 additions & 0 deletions src/lib/utils/provider-group.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import {
normalizeProviderGroup,
normalizeProviderGroupTag,
parseProviderGroups,
resolveBillingProviderGroups,
resolveProviderGroupsWithDefault,
} from "./provider-group";

Expand Down Expand Up @@ -33,4 +34,33 @@ describe("provider-group utils", () => {
expect(parseProviderGroups(null)).toEqual([]);
expect(parseProviderGroups(" ")).toEqual([]);
});

test("计费分组应取用户分组与已选供应商标签的交集", () => {
expect(
resolveBillingProviderGroups("cus_gpt,gpt_test", "cus_claude_pro,cus_grok,gpt_test,mimo")
).toEqual(["gpt_test"]);
});

test("计费分组应保留用户分组声明顺序", () => {
expect(resolveBillingProviderGroups("group-b,group-a", "group-a,group-b")).toEqual([
"group-a",
"group-b",
]);
});

test("通配分组应按供应商标签解析倍率", () => {
expect(resolveBillingProviderGroups("group-b,group-a", "*")).toEqual(["group-b", "group-a"]);
});

test("显式匹配分组应优先于通配分组", () => {
expect(resolveBillingProviderGroups("group-b,group-a", "*,group-a")).toEqual(["group-a"]);
});

test("未分组供应商在通配访问下应使用 default 计费分组", () => {
expect(resolveBillingProviderGroups(null, "*")).toEqual(["default"]);
});

test("无交集且无通配权限时不应选择无关分组倍率", () => {
expect(resolveBillingProviderGroups("group-b", "group-a")).toEqual([]);
});
});
30 changes: 30 additions & 0 deletions src/lib/utils/provider-group.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,33 @@ export function resolveProviderGroupsWithDefault(value: unknown): string[] {

return groups;
}

/**
* Resolve the provider groups that should participate in billing.
*
* Explicit user/key groups are intersected with the selected provider's tags,
* preserving the user/key declaration order. A wildcard grants access to all
* providers but only falls back to the provider's own tag order when there is
* no explicit matching group.
*/
export function resolveBillingProviderGroups(
providerGroupTag: unknown,
userGroupsValue: unknown
): string[] {
const providerGroups = resolveProviderGroupsWithDefault(providerGroupTag);
const userGroups = parseProviderGroups(userGroupsValue);
const providerGroupSet = new Set(providerGroups);

const explicitMatches = userGroups.filter(
(group) => group !== PROVIDER_GROUP.ALL && providerGroupSet.has(group)
);
if (explicitMatches.length > 0) {
return explicitMatches;
}

if (userGroups.includes(PROVIDER_GROUP.ALL)) {
return providerGroups;
}

return [];
}
171 changes: 171 additions & 0 deletions tests/unit/proxy/provider-selector-select-provider-by-type.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import type { Provider } from "@/types/provider";
import { ProxyProviderResolver } from "@/app/v1/_lib/proxy/provider-selector";

const findAllProvidersMock = vi.hoisted(() => vi.fn<[], Promise<Provider[]>>());
const getGroupCostMultiplierMock = vi.hoisted(() => vi.fn());
const checkAndTrackProviderSessionMock = vi.hoisted(() => vi.fn());

vi.mock("@/repository/provider", () => {
return {
Expand All @@ -11,6 +13,16 @@ vi.mock("@/repository/provider", () => {
};
});

vi.mock("@/repository/provider-groups", () => ({
getGroupCostMultiplier: getGroupCostMultiplierMock,
}));

vi.mock("@/lib/rate-limit", () => ({
RateLimitService: {
checkAndTrackProviderSession: checkAndTrackProviderSessionMock,
},
}));

describe("ProxyProviderResolver.selectProviderByType - /v1/models 分组隔离", () => {
beforeEach(() => {
vi.clearAllMocks();
Expand Down Expand Up @@ -89,3 +101,162 @@ describe("ProxyProviderResolver.selectProviderByType - /v1/models 分组隔离",
expect(provider?.id).toBe(inGroup.id);
});
});

describe("ProxyProviderResolver.ensure - 分组倍率", () => {
beforeEach(() => {
vi.clearAllMocks();
});

test("按当前供应商与 Key 分组交集解析倍率", async () => {
const provider = {
id: 56,
name: "gpt-test-provider",
isEnabled: true,
providerType: "openai-compatible",
groupTag: "cus_gpt,gpt_test",
weight: 1,
priority: 0,
costMultiplier: 1,
limitConcurrentSessions: 0,
} as unknown as Provider;

getGroupCostMultiplierMock.mockResolvedValueOnce(10);

const findReusableSpy = vi
.spyOn(ProxyProviderResolver as never, "findReusable" as never)
.mockResolvedValue(null as never);
const pickRandomProviderSpy = vi
.spyOn(ProxyProviderResolver as never, "pickRandomProvider" as never)
.mockResolvedValue({
provider,
context: {
totalProviders: 1,
enabledProviders: 1,
targetType: "openai-compatible",
requestedModel: "gpt-5.5",
groupFilterApplied: true,
userGroup: "cus_claude_pro,cus_grok,gpt_test,mimo",
beforeHealthCheck: 1,
afterHealthCheck: 1,
priorityLevels: [0],
selectedPriority: 0,
candidatesAtPriority: [],
},
} as never);

const setGroupCostMultiplier = vi.fn();
const session = {
provider: null as Provider | null,
sessionId: null,
authState: {
user: { providerGroup: "cus_claude_pro,cus_grok,gpt_test,mimo" },
key: { providerGroup: "cus_claude_pro,cus_grok,gpt_test,mimo" },
},
setProvider(selected: Provider | null) {
this.provider = selected;
},
setLastSelectionContext: vi.fn(),
getLastSelectionContext: vi.fn(() => null),
setGroupCostMultiplier,
addProviderToChain: vi.fn(),
getOriginalModel: vi.fn(() => "gpt-5.5"),
} as unknown as Parameters<typeof ProxyProviderResolver.ensure>[0];

try {
await expect(ProxyProviderResolver.ensure(session)).resolves.toBeNull();
} finally {
findReusableSpy.mockRestore();
pickRandomProviderSpy.mockRestore();
}

expect(getGroupCostMultiplierMock).toHaveBeenCalledWith("gpt_test");
expect(setGroupCostMultiplier).toHaveBeenCalledWith(10);
});

test("故障切换后应按最终供应商重新解析倍率", async () => {
const firstProvider = {
id: 1,
name: "group-a-provider",
providerType: "openai-compatible",
groupTag: "group-a",
weight: 1,
priority: 0,
costMultiplier: 1,
limitConcurrentSessions: 1,
} as unknown as Provider;
const fallbackProvider = {
...firstProvider,
id: 2,
name: "group-b-provider",
groupTag: "group-b",
limitConcurrentSessions: 0,
} as Provider;

getGroupCostMultiplierMock.mockResolvedValueOnce(10);
checkAndTrackProviderSessionMock
.mockResolvedValueOnce({
allowed: false,
count: 1,
referenced: false,
reason: "limit reached",
})
.mockResolvedValueOnce({
allowed: true,
count: 1,
referenced: false,
});

const context = {
totalProviders: 2,
enabledProviders: 2,
targetType: "openai-compatible",
requestedModel: "gpt-5.5",
groupFilterApplied: true,
userGroup: "group-a,group-b",
beforeHealthCheck: 2,
afterHealthCheck: 2,
priorityLevels: [0],
selectedPriority: 0,
candidatesAtPriority: [],
};

const findReusableSpy = vi
.spyOn(ProxyProviderResolver as never, "findReusable" as never)
.mockResolvedValue(null as never);
const pickRandomProviderSpy = vi
.spyOn(ProxyProviderResolver as never, "pickRandomProvider" as never)
.mockResolvedValueOnce({ provider: firstProvider, context } as never)
.mockResolvedValueOnce({ provider: fallbackProvider, context } as never);

const setGroupCostMultiplier = vi.fn();
const session = {
provider: null as Provider | null,
sessionId: "session-1",
authState: {
user: { providerGroup: "group-a,group-b" },
key: { providerGroup: "group-a,group-b" },
},
setProvider(selected: Provider | null) {
this.provider = selected;
},
setLastSelectionContext: vi.fn(),
getLastSelectionContext: vi.fn(() => context),
setGroupCostMultiplier,
addProviderToChain: vi.fn(),
getOriginalModel: vi.fn(() => "gpt-5.5"),
recordProviderSessionRef: vi.fn(),
} as unknown as Parameters<typeof ProxyProviderResolver.ensure>[0];

try {
await expect(ProxyProviderResolver.ensure(session)).resolves.toBeNull();
} finally {
findReusableSpy.mockRestore();
pickRandomProviderSpy.mockRestore();
}

expect(checkAndTrackProviderSessionMock).toHaveBeenCalledTimes(2);
expect(getGroupCostMultiplierMock).toHaveBeenCalledTimes(1);
expect(getGroupCostMultiplierMock).toHaveBeenCalledWith("group-b");
expect(setGroupCostMultiplier).toHaveBeenCalledWith(10);
});
});
Loading