[Feature] Pass-through endpoints — proxy provider-specific APIs beyond chat completions

[carlosatta]

Principle

Routerly must be a transparent drop-in replacement for any LLM provider. A user who today calls OpenAI, Anthropic, Ollama, etc. must be able to swap the base URL for Routerly's without changing a single line of code. Everything must work: current endpoints, future endpoints, future API versions.

This also means Routerly should never need to be updated just because a provider adds new APIs or changes endpoints — the proxy passes everything through, intercepting only what it needs.

Current problem

Routerly only handles paths it explicitly knows about (/v1/chat/completions, Anthropic messages). Any other path is not forwarded. This breaks the drop-in principle:

• A user who calls the Embeddings API must maintain two base URLs in their code (one for Routerly, one for the provider directly)
• • When OpenAI/Anthropic/Ollama add new endpoints, Routerly must be updated to support them
• • • Same applies to any custom provider

Solution

Transparent default: any path not explicitly handled by Routerly is forwarded to the upstream provider, replacing only the API key. No config needed.

Routing depends on the provider configured in the project, not on the path. Examples:

# Project with OpenAI provider
POST /v1/chat/completions  →  routing engine  →  provider    (handled by Routerly)
POST /v1/*                 →  OpenAI /v1/*                   (pass-through)

# Project with Anthropic provider
POST /v1/messages          →  routing engine  →  provider    (handled by Routerly)
POST /v1/*                 →  Anthropic /v1/*                (pass-through)

# Routerly internal management API — excluded from proxy
GET|POST /api/*            →  Routerly internal              (unchanged)

Routerly intercepts only the paths it actively manages. Everything else is forwarded unchanged — today and for any future provider updates, without Routerly needing to be touched.

Scope

• All providers (OpenAI, Anthropic, Ollama, custom providers)
• • Routerly's internal management APIs (/api/*, dashboard) must not be affected by the pass-through mechanism
• • • No config required: transparent behavior is the default

Acceptance criteria

• Any path not explicitly handled is forwarded to the project's upstream provider
• - [ ] The provider API key is substituted automatically
• - [ ] Behavior of explicitly handled paths (chat completions, messages) remains unchanged
• - [ ] Routerly's internal APIs (/api/*) are never proxied
• - [ ] Works for all configurable providers

[carlosatta]

Implementation — PR #106

Approach

The pass-through is implemented as a global Fastify not-found handler (setNotFoundHandler). Any path not matched by an explicit route lands here instead of returning a default 404.

Request flow

Client → Routerly /v1/embeddings
  │
  ├─ auth preHandler: validates Bearer token → decorates request.project
  │
  ├─ no route matches → passthroughHandler
  │
  ├─ guard: path is /api/*, /dashboard*, /, /health → 404 (never proxied)
  │
  ├─ pickUpstreamModel: matches body.model against project's configured models,
  │   falls back to the first configured model
  │
  ├─ buildUpstreamUrl: origin(model.endpoint) + requestUrl
  │   e.g. "https://api.openai.com" + "/v1/embeddings"
  │
  ├─ buildUpstreamHeaders: clones headers, drops host / content-length /
  │   connection / authorization / x-api-key, injects upstream key
  │   (x-api-key for Anthropic, Authorization: Bearer for everything else)
  │
  ├─ fetch(upstreamUrl, { method, headers, body })
  │   body: raw Buffer for non-JSON content, JSON.stringify for parsed objects
  │
  └─ response streamed back to client via Readable.fromWeb(upstream.body)
      status and headers (excluding hop-by-hop) copied verbatim

Files changed

File	Change
packages/service/src/routes/passthrough.ts	New — pure helpers + handler
packages/service/src/plugins/auth.ts	Extracted resolveProjectByToken() as a shared exported helper
packages/service/src/server.ts	* content-type parser for non-JSON bodies + setNotFoundHandler
packages/service/src/routes/passthrough.test.ts	22 new tests
packages/service/src/server.telemetry.test.ts	Updated mock stubs

Known limitations

• Providers with a sub-path in the endpoint (e.g. Gemini /v1beta/openai) are mapped using origin only — the sub-path is not preserved. Can be refined later if needed.
• Ollama's native /api/* endpoints cannot be proxied because /api/* is Routerly's reserved management namespace (per the issue). Ollama's OpenAI-compatible /v1/* endpoints work correctly.
• No token/cost tracking for pass-through traffic (response formats are arbitrary).