Skip to content

feat: add model routing and composer compatibility#3

Open
kksp993 wants to merge 2 commits into
GreyGunG:mainfrom
kksp993:grok-cli-composer-routing
Open

feat: add model routing and composer compatibility#3
kksp993 wants to merge 2 commits into
GreyGunG:mainfrom
kksp993:grok-cli-composer-routing

Conversation

@kksp993

@kksp993 kksp993 commented Jul 12, 2026

Copy link
Copy Markdown

Summary

Hi, thank you for maintaining this project. While using grokbuild-proxy with multiple credentials, I ran into two practical compatibility gaps that seem useful to support in the project itself:

  • Some upstream models are subscription/account-class sensitive, but the current load balancer treats all enabled credentials as interchangeable.
  • grok-composer-2.5-fast appears to be more reliable when requests match the official Grok CLI chat-completions wire shape.

This PR adds two opt-in building blocks to improve those cases while preserving existing behavior by default.

Current behavior / limitation

1. Credentials cannot be restricted per model

At the moment, model requests are selected through the normal credential load balancer. That works well when all credentials have the same capabilities, but it can be problematic when different credentials have different model access.

For example, if only a subset of credentials can access a subscription-gated model, requests for that model may be routed to an incompatible credential and fail, even though a compatible credential exists in the pool.

2. Composer model compatibility

For grok-composer-2.5-fast, upstream behavior appears to differ depending on the request shape. The official Grok CLI sends composer generation requests through /chat/completions with Grok CLI-style headers and a stable agent identifier.

This PR keeps the public proxy API behavior intact, but adapts composer requests internally to better match that official client shape.

What this PR changes

Commit 1: model-based credential routing

Adds optional config-driven model routing:

model_routing:
  enabled: true
  routes:
    - models:
        - "grok-composer-2.5-fast"
      credential_ids:
        - "credential-id-here"

Behavior:

  • If model routing is disabled, existing behavior is unchanged.
  • If a model does not match a route, existing load balancing is unchanged.
  • If a model matches a route, only the configured credential IDs are considered.
  • If none of the configured credentials are available, the request fails instead of falling back to unrelated credentials.

This avoids sending subscription-gated model traffic to incompatible credentials.

Commit 2: Grok CLI-compatible composer adapter

Adds:

  • Stable persisted x-grok-agent-id support.
  • A composer-specific internal adapter that:
    • accepts the existing Responses API request shape from clients,
    • converts composer requests to upstream /chat/completions,
    • applies Grok CLI-style request headers,
    • converts the upstream chat-completions response back to Responses API shape.

This keeps downstream compatibility while matching the upstream request shape that composer models expect.

Backward compatibility

This should be backward compatible:

  • model_routing is disabled by default.
  • Unmatched models keep the existing load balancer behavior.
  • Existing /v1/responses callers can continue using the same public API.
  • The generated agent ID is stored under the configured data directory and reused.

Test plan

Added/updated tests for:

  • model routing config validation,
  • routed credential selection,
  • no fallback to unrelated credentials for matched routes,
  • normal load balancing for unmatched models,
  • agent ID generation/reuse,
  • upstream header propagation,
  • composer request adaptation.

Tested locally with:

go test ./cmd/grokbuild-proxy ./internal/config ./internal/proxy ./internal/upstream

Notes

I tried to keep this PR configuration-driven and avoid introducing credential schema changes. The goal is to give operators a safe way to separate credentials by model capability without changing existing import flows or default runtime behavior.

kksp993 added 2 commits July 12, 2026 20:57
Allow operators to restrict selected models to explicit credential allowlists. This keeps subscription-gated model traffic on compatible credentials while preserving the existing load balancer for unmatched models.
Persist a stable Grok agent identifier and route composer responses requests through a CLI-compatible chat completions adapter. This matches the official client wire shape for composer models while keeping the public Responses API behavior intact.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant