Skip to content

feat: Add LM Studio provider with dynamic model discovery#744

Open
athul0rameshan9 wants to merge 6 commits into
THU-MAIC:mainfrom
athul0rameshan9:feat/lmstudio-model-discovery
Open

feat: Add LM Studio provider with dynamic model discovery#744
athul0rameshan9 wants to merge 6 commits into
THU-MAIC:mainfrom
athul0rameshan9:feat/lmstudio-model-discovery

Conversation

@athul0rameshan9

Copy link
Copy Markdown

Summary

Add LM Studio as a built-in provider and implement a 'Fetch Models' button that auto-discovers available models from any OpenAI-compatible local server, eliminating the need to manually specify models in the env file.

Related Issues

Fixes #730

Changes

  • Register lmstudio as a built-in provider (localhost:1234/v1, no API key required)
  • Add /api/list-models endpoint to fetch models from OpenAI-compatible /v1/models APIs
  • Add 'Fetch Models' button to provider settings panel for all OpenAI-type providers
  • Add LMSTUDIO_BASE_URL / LMSTUDIO_MODELS env var support in server provider config
  • Add i18n translations (en-US, zh-CN)

Type of Change

  • New feature (non-breaking change that adds functionality)

Verification

  • My changes do not introduce new warnings
  • TypeScript compiles without errors (tsc --noEmit)

@wyuc wyuc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, and welcome — a clean, well-structured first contribution (nice that you covered every locale). A few things to address before merge; two are security-related.

🔴 Server API key can leak to a client-supplied hostapp/api/list-models/route.ts
For a server-managed provider configured with only an API key (e.g. OPENAI_API_KEY set, no OPENAI_BASE_URL), resolveBaseUrl() returns undefined, so resolveBaseUrl(...) || clientBaseUrl falls back to the client-supplied baseUrl while resolveApiKey() still returns the server secret. The request then sends Authorization: Bearer <server key> to an arbitrary client-controlled host. For managed providers, please ignore clientBaseUrl entirely and use only the server/default URL when a server key is attached.

🔴 Missing SSRF guardapp/api/list-models/route.ts
The other routes that fetch a client-supplied URL (app/api/generate/voice, azure-voices, generate/video, proxy-media, parse-pdf) all call validateUrlForSSRF() from @/lib/server/ssrf-guard before fetching. This new route fetches after only new URL() parsing, so a crafted baseUrl can make the server probe internal addresses, and the upstream error body is returned to the caller. Please run the same validateUrlForSSRF() before building/fetching modelsUrl. This matters most for hosted / multi-tenant deployments.

🟡 Success-response shape mismatch breaks the featurecomponents/settings/provider-config-panel.tsx
apiSuccess({ models }) serializes as { success: true, models: [...] } (the helper spreads, it does not nest under data). The client reads data.data?.models, which is always undefined, so even a successful fetch shows the failure message. Read data.models instead.

Once the two security items are addressed the rest looks good. Thanks again for the contribution!

Add LM Studio as a built-in provider and implement a 'Fetch Models' button
that queries the /v1/models endpoint to auto-discover available models from
any OpenAI-compatible local server (LM Studio, Ollama, etc.).

Changes:
- Register lmstudio as a built-in provider (localhost:1234/v1, no API key)
- Add /api/list-models endpoint to fetch models from OpenAI-compatible APIs
- Add 'Fetch Models' button to provider settings for OpenAI-type providers
- Add LMSTUDIO_BASE_URL/LMSTUDIO_MODELS env var support
- Add i18n translations (en-US, zh-CN)

Users no longer need to manually specify models in the env file when using
LM Studio - they can discover and select models directly from the app UI.
@athul0rameshan9 athul0rameshan9 force-pushed the feat/lmstudio-model-discovery branch from b8b74a9 to 83fdace Compare June 17, 2026 08:04
…fix response shape

- Sanitize client inputs for managed providers (ignore client key/baseUrl)
  to prevent leaking server API keys to attacker-controlled hosts
- Add validateUrlForSSRF() check on client-supplied base URLs, matching
  the pattern used by tts/voice/video routes
- Fix response shape: read data.models instead of data.data?.models to
  match apiSuccess() output format
@athul0rameshan9 athul0rameshan9 force-pushed the feat/lmstudio-model-discovery branch from 83fdace to 1ddb0bd Compare June 17, 2026 08:10
@athul0rameshan9

Copy link
Copy Markdown
Author

Thanks for the review! All three items have been addressed:

  • 🔴 API key leak: Managed providers now ignore all client-supplied credentials (isServerConfiguredProvider check, matching the pattern in tts/voice/video routes)
  • 🔴 SSRF guard: Added validateUrlForSSRF() on client-supplied base URLs before fetching
  • 🟡 Response shape: Client now reads data.models instead of data.data?.models

Also rebased onto latest main. Ready for re-review!

@athul0rameshan9 athul0rameshan9 requested a review from wyuc June 18, 2026 12:49
@wyuc

wyuc commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Review: /api/list-models + LM Studio provider

Thanks for the PR — the model auto-discovery is a nice addition, and the SSRF guard + managed-provider key/baseUrl isolation are good to see. One blocking issue and a couple of follow-ups before merge.

🔴 P1 (blocking) — Redirect-based SSRF bypass · app/api/list-models/route.ts

The outbound fetch(modelsUrl, …) uses the default redirect: 'follow', but validateUrlForSSRF() only validates the initial user-supplied baseUrl. An attacker can supply a public baseUrl that returns a 3xx redirect to an internal target (e.g. cloud metadata http://169.254.169.254/…, localhost, or an internal service); fetch transparently follows it and the redirect target is never re-validated — so the SSRF guard is bypassed and the upstream body is returned to the caller.

This is reachable unauthenticated on deployments where ACCESS_CODE is unset (the default for the public demo and typical self-host).

The repo already has the right pattern for this — please mirror it:

/api/list-models is currently the only user-URL-fetching route missing this protection.

🟡 P2 — defaultBaseUrl fallback · app/api/list-models/route.ts + components/settings/index.tsx

Two issues on the same fallback path:

  • Security: only safeClientBaseUrl goes through validateUrlForSSRF. The || provider?.defaultBaseUrl branch is fetched without any SSRF check. It's a hardcoded constant today (so low risk), but the guard should run against the resolved baseUrl, not just the raw client input. (The middle || safeClientBaseUrl term is also dead code — resolveBaseUrl already returns it for unmanaged providers.)
  • Correctness: for a fresh LM Studio setup with the Base URL field left blank, the fetch succeeds via defaultBaseUrl but only the models are persisted, not the base URL. Since keyless providers are considered configured only when baseUrl is set, the provider still won't appear in the active model list until the user manually types the URL. Either persist the effective default base URL, or require it before fetching.

🟡 P2 — Upstream error body echoed verbatim · app/api/list-models/route.ts

On a non-OK upstream response the route returns Provider returned ${status}: ${errorText} (the full raw upstream body) to the client. Combined with the P1 this is the exfiltration channel, but it's also an independent info leak. Other routes (verify-model, verify-image-provider) return sanitized/generic messages and log the detail server-side — please do the same here.

🔵 P3 (nit) — SSRF gating divergence

This route validates unconditionally, whereas most sibling routes gate the check on NODE_ENV === 'production'. Unconditional is the more secure choice (and there's the ALLOW_LOCAL_NETWORKS=true escape hatch), but note it means a local LM Studio at localhost:1234 is blocked unless that env var is set — which slightly undercuts the feature's main use case. A short code comment explaining the trade-off would help.


Verified OK

Managed-provider key isolation (client key/baseUrl suppressed for admin-owned providers) ✅ · AbortController 10s timeout with clearTimeout on all paths ✅ · localeCompare guarded by if (model.id) ✅.

The P1 is the only hard blocker. Happy to re-review once the redirect handling is in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Ability to select model in app

3 participants