Add lemon-mlx-engine as a wrapped-server backend (rebase of #2010)#2472
Open
bong-water-water-bong wants to merge 58 commits into
Open
Add lemon-mlx-engine as a wrapped-server backend (rebase of #2010)#2472bong-water-water-bong wants to merge 58 commits into
bong-water-water-bong wants to merge 58 commits into
Conversation
Wires lemon-mlx-engine (https://github.com/lemonade-sdk/lemon-mlx-engine) as a new recipe/backend pair on top of the existing WrappedServer abstraction. The engine is a subprocess that exposes the OpenAI chat/completions API, so this PR mostly adds registration glue: - `MlxServer` class (src/cpp/{include,server}/.../mlx_server.{h,cpp}) forwarding chat/completions + completions, unsupported for responses. Auto-download model refs go directly to the subprocess as positional args (MLX resolves HF repo-ids itself), with ROCm builds picking the correct per-arch release asset via `SystemInfo::get_rocm_arch()`. - Registered in `try_get_spec_for_recipe`, `backend_versions.json` (pinned at `b1013-stable` for metal/rocm/cpu), `runtime_config` (s_backend_names + s_selectable_backends), `model_types` (`get_device_type_from_recipe` → GPU default, CPU override in load()), and the RECIPE_DEFS support matrix in `system_info.cpp` (Apple Silicon metal, Linux AMD rocm on gfx1150/1151/110X/120X, CPU fallback). - Two sample entries in `server_models.json` (`Qwen3-0.6B-MLX` suggested, `Qwen3-4B-MLX` opt-in) plus capabilities catalog + `--mlx-engine` backend flag in the test harness so `python test/server_llm.py --wrapped-server mlx-engine --backend cpu` is a valid invocation. Smoke: `GET /api/v1/system-info` on Linux gfx1151 reports mlx-engine with cpu=installable and rocm=installable pointing at `mlx-engine-b1013-stable-ubuntu-rocm-stable-gfx1151-x64.zip`; metal is correctly flagged unsupported on Linux. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Geramy asked to rebrand the backend identifier from `mlx-engine` to `lemon-mlx` — "sounds more market friendly and cool." Applied across the wrapped-server wiring, system-info metadata, recipe options, and test harness utilities. Release-asset filenames keep the existing `mlx-engine-<version>-*.zip` prefix because that is what the lemonade-sdk/lemon-mlx-engine CI currently produces; backend-name rename is independent of artifact naming. Changes: - `RECIPE_DEFS` entries: `"mlx-engine"` → `"lemon-mlx"` for metal, rocm, and cpu variants. - `s_backend_names` and `s_selectable_backends` in runtime_config. - `backend_versions.json` key: `mlx-engine` → `lemon-mlx`. - `server_models.json` recipe strings. - `WrappedServer` constructor tag: `"lemon-mlx"`. - `RecipeOptions` keys: `mlx_engine_backend` → `lemon_mlx_backend`, `mlx_engine_args` → `lemon_mlx_args` — unmerged PR so no migration concern. - test/utils/capabilities.py + server_base.py: identifier + recipe key matches. - model_types::get_device_type_from_recipe dispatch string. - backend_utils::try_get_spec_for_recipe dispatch string. - Log-line and error-message strings for operator readability. Verification: - `cmake --build --preset default --target lemond` clean (GCC, gfx1151 ROCm-stable userspace). - `./build/lemond --version` → 10.2.0. Version bump from `b1013-stable` to the new stable+preview releases is left as a follow-up once Geramy's CI completes (he flagged it would finish ~30 min after the original ping).
Geramy landed b1005-preview on lemon-mlx-engine 2026-04-23 16:54Z. Wire it alongside the existing stable pin (b1013-stable) following the same 'rocm' / 'rocm-preview' pattern llamacpp already uses in this file. Users on the preview channel pick up newer engine builds at kernel-level risk; stable users keep the current pinned behaviour. No runtime-code change needed — server_info.cpp consults this file by key; adding '*-preview' keys is transparent to it.
# Conflicts: # CMakeLists.txt # src/app/src/renderer/recipes/recipeOptionsConfig.ts # src/cpp/server/recipe_options.cpp # src/cpp/server/router.cpp # src/cpp/server/runtime_config.cpp # src/cpp/server/system_info.cpp
The preview pins were stale at b1005-preview while stable was at b1013-stable. Upstream lemon-mlx-engine cuts paired stable/preview tags at every build, so a preview pin lagging stable by 8 builds was inconsistent with the release scheme.
The Ubuntu CPU asset ships libopenblas.so.0, liblapacke.so.3, and
libgfortran.so.5 next to the server binary, but the binary has no
DT_RUNPATH, so without LD_LIBRARY_PATH ld.so falls back to system
search and exits 127. The LD_LIBRARY_PATH plumbing already existed
but was gated to the rocm variant only; the cpu variant has the same
bundling pattern. Extend the bundle-dir lookup to all Linux variants
(metal is macOS-only and can't reach this block).
Fixes the "Test .deb - lemon-mlx" CI job, which loaded a Qwen3-0.6B
MLX model and saw the wrapped server die with:
./bin/lemon-mlx/cpu/server: error while loading shared libraries:
libopenblas.so.0: cannot open shared object file
lemon-mlx process has terminated with exit code: 127
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Upstream lemon-mlx URL-joins the model argument onto huggingface.co/api/models/ so an absolute filesystem path produces a 404 like huggingface.co/api/models//home/.../hf-cache/hub/models--mlx-community--Qwen3-0.6B-4bit/revision/main Swap the priority: pass model_info.checkpoint() (HF repo-id) first; fall back to resolved_path() only when checkpoint is empty (rare custom-model recipes). HF_HOME is honored by the lemon-mlx process, so a previously downloaded model is still resolved from the local cache without an additional download. Fixes the second CI failure on "Test .deb - lemon-mlx" (after the previous LD_LIBRARY_PATH fix in d091d88 closed the libopenblas exit 127). The chat-completions test was loading Qwen3-0.6B-MLX and seeing the wrapped server die with exit code 1 + a 404 on a malformed HF API URL. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…at/mlx-engine-backend
lemon-mlx-engine ships no Linux arm64 asset; only ubuntu-cpu-x64.zip exists for Linux. Without this guard, get_install_params() would silently download and attempt to run an x64 binary on a Linux arm64 host. Fail fast with a clear message instead. Closes Codex P2 review comment on mlx_server.cpp.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lemonade-sdk#1975) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* custom collection models possible - besides new model also new collections - lean UI with current design - can be exported and imported * polish custom collection UI - slightly adapt readme.md - Move the custom collection utility test under test/app * fix render bug for normal model setup - deleted old files and adapted for npm test * adapted to fitting it properly into new omni-router concept pr1842 * 2nd stage adapting towards new concept * cleaned UI and finilize omni model integration * visibility and selection and UI fixes
* Rebrand OmniRouter to Lemonade Omni * Refine documentation * docs: relocate custom Omni Model sections and align example LLM - Move "Custom Omni Models" how-to from docs/dev/lemonade-omni.md to docs/guide/configuration/custom-models.md as a sibling of the CLI "Register an omni collection" subsection. lemonade-omni.md now links there. - Move "Testing custom Omni Models" (automated + manual smoke test) from lemonade-omni.md to docs/dev/app.md, adjacent to the existing "Testing the Rust host" section. - examples/lemonade_tools.py: default LLM_MODEL → Qwen3.5-4B-MTP-GGUF to match the LMN-Lite-Omni-5.5B LLM component (the comment already said "match LMN-Lite-Omni-5.5B"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Rename omni model prefix LMN → LMX (Lemonade Mix) Switch to the LMX-Omni-<xB>-<class> naming pattern: - LMN-Halo-Omni-52B → LMX-Omni-52B-Halo - LMN-Lite-Omni-5.5B → LMX-Omni-5.5B-Lite LMX stands for "Lemonade Mix". Updated the docs decoder ring, example script references, model_sizes.py usage example, and the server_models.json registry keys. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs/test: post-rebrand consistency sweep - Fix menu paths: the desktop app's menu bar is File/View/Help, not "Lemonade". Updated docs/guide/configuration/custom-models.md and docs/dev/app.md to say "File > New Omni Model > Manually", and examples/lemonade_tools.py to point to the actual download path (Model Manager > Lemonade Omni > LMX-Omni-5.5B-Lite > Download). - Tighten "desktop apps model list" → "Lemonade desktop app's Model Manager" in docs/dev/lemonade-omni.md. - Update the customCollections test fixture from the legacy "Lite Collection" name to the current "LMX-Omni-5.5B-Lite" so the built-in-template assertion reflects today's primary name. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * model_sizes: forward HF_TOKEN on range fetches fetch_repo() already attached the Authorization header from HF_TOKEN, but fetch_range() did not, so the safetensors/GGUF header-parsing fallback path silently 401'd against gated/private repos even when the user had set HF_TOKEN. Result: param counts downgraded to unknown when the HF API didn't already report gguf.total / safetensors.total. Extract an hf_headers() helper and use it from both call sites so the auth concern lives in one place and future HF fetches inherit it. Reported in PR lemonade-sdk#1973 review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Rename desktop category label "Lemonade Omni" → "Lemonade" Updates both the Model Manager category label and the recipe display name for collection.omni, plus the two doc mentions of the category. "Lemonade" alone reads cleaner now that Omni Models are the only content under this category. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: missing file size in custom models * Lock the models cache before modifying
…view pins Blockers: - test/utils/capabilities.py: set completions_streaming=False for lemon-mlx; lemon-mlx-engine does not implement SSE streaming on /v1/completions - mlx_server.cpp: guard forward_streaming_request against /v1/completions direct calls that bypass the router capability check P2 (fl0rianr 2026-05-22): - system_info.cpp: split lemon-mlx cpu entry — Linux x86_64 only, macOS arm64 only; matches installer asset reality and get_install_params() guards - backend_versions.json: remove unreachable *-preview pins (metal/rocm/cpu); no config path selects lemon-mlx:*-preview today - README.md: make CPU row explicit: Linux x86_64 / macOS arm64 (beta) - mlx_server.cpp: replace custom tokenize_quoted_args with std::istringstream, matching the pattern in fastflowlm_server.cpp and vllm_server.cpp
… error for streaming completions - resolve_mlx_backend: auto-select rocm only for supported GPU families (gfx1150, gfx1151, gfx110X, gfx120X), fall back to cpu for RDNA2/gfx103X - prepare_request: always erase max_completion_tokens after handling it - forward_streaming_request: write SSE error + sink.done() instead of throw Co-authored-by: multica-agent <github@multica.ai>
…Intel Mac guard - Rename MlxServer::SPEC to spec (snake_case convention) - Move namespace fs and using namespace lemon::utils inside namespace block - Alphabetize include blocks in mlx_server.cpp - Add architecture guard for Intel macOS (arm64-only for MLX Metal/CPU) Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
…DE_MOCK_HARDWARE guard 1. model_manager.cpp: Move fs::file_size() calls outside models_cache_mutex_ in update_model_in_cache(). Copy resolved paths under lock, release, then calculate size. Re-lock briefly to write the rounded size if the cache entry is still valid. 2. system_info.cpp: Add LEMONADE_MOCK_HARDWARE=1 env var guard to get_system_info_with_cache(). Mock hardware_info.json is now only loaded when the env var is explicitly set, preventing unintended mock loading. 3. test_flm_status.py: Set LEMONADE_MOCK_HARDWARE=1 in the environment when starting the server with mock hardware. Co-authored-by: multica-agent <github@multica.ai>
…VRAM > less) Prior behavior: get_rocm_arch() returned the first available AMD GPU, which is always the iGPU (added first in the devices array). On systems with a gfx1036 iGPU + gfx1201 dGPU, this returned 'gfx1036' — causing all ROCm backends to download binaries for the wrong architecture, install TheRock for the wrong GPU, and silently fall back to CPU. New behavior: iterates ALL AMD GPUs and returns the highest-scoring architecture using a VRAM-based ranking: - Primary factor: VRAM capacity (more VRAM = better for LLM inference) - Tiebreaker: dGPU bonus (GPUs with >= 4 GB VRAM are likely discrete) This mirrors get_cuda_arch() which also iterates all NVIDIA GPUs and picks the highest compute capability. Fixes: lemonade-sdk#2296, lemonade-sdk#2319 (GPU selection), helps lemonade-sdk#1787, lemonade-sdk#1991, lemonade-sdk#2002, lemonade-sdk#2371
… on hybrid AMD+NVIDIA laptops On systems with both AMD iGPU and NVIDIA dGPU, vLLM's platform detection finds both 'cuda' and 'rocm' plugins and refuses to start with: RuntimeError: Only one platform plugin can be activated, but got: ['cuda', 'rocm'] Set VLLM_TARGET_DEVICE=rocm in the vLLM subprocess environment so the ROCm launcher activates only the ROCm platform plugin and skips the CUDA/NVML probe. Closes lemonade-sdk#1863 Co-Authored-By: Claude <noreply@anthropic.com>
…memory crash On hybrid AMD+NVIDIA systems, llama.cpp's ggml memory budgeter counts both GPUs' VRAM pools, then tries to allocate the full model on the ROCm device — exceeding available memory and failing with: ggml_backend_cuda_buffer_type_alloc_buffer: cudaMalloc failed: out of memory Setting CUDA_VISIBLE_DEVICES="" hides the NVIDIA GPU from ggml's memory calculation while the ROCm backend discovers AMD GPUs through ROCr directly. User-confirmed workaround on Strix Halo + RTX 3090 Ti. Closes lemonade-sdk#2074 Co-Authored-By: Claude <noreply@anthropic.com>
…nds, memory guard Three-layer defense against RyzenAI/NPU inference hangs on memory-constrained systems (e.g. Qwen3-14B-Hybrid on 24 GB RAM): B1+B2: Apply default_timeout_seconds_ (300s) fallback to HttpClient::post() and post_stream(), matching the existing safety net in get(). Previously post() passed timeout_seconds=0 directly to CURLOPT_TIMEOUT, meaning 0=infinite. A hung backend would block the HTTP worker thread forever, freezing the UI. C2: Start the backend watchdog in every backend's load() method (llama.cpp, vLLM, RyzenAI, Whisper, Moonshine, SD, Kokoro). Only FastFlowLM had it before. All backends already call stop_backend_watchdog() in unload(). The watchdog probes /health and resets a hung backend after the grace period (default 90s). A2: Pre-reject NPU model loads when auto-tune can only reserve the fallback ctx_size (AUTO_CTX_FALLBACK=4096) and the model exceeds 10 GB. On memory-constrained systems, the NPU driver's runtime overhead pushes past the limit during first inference, causing the backend to hang. Rejecting early gives a clear error instead of a silent hang. Closes lemonade-sdk#1151 Co-Authored-By: Claude <noreply@anthropic.com>
… cooldown, TDR warning Three fixes for Vulkan backend hangs on Windows with AMD/Intel iGPUs (lemonade-sdk#2000): 1. Default --no-flash-attn on Windows+Vulkan. Flash attention on Vulkan Windows has caused reproducible hangs since llama.cpp b7064 (ggml#17297). Users can opt back in with --flash-attn in llamacpp_args. 2. 2-second GPU driver cooldown in ProcessManager::terminate() on Windows, matching the existing Linux behavior. Without this, orphaned Vulkan command buffers leave the GPU pegged at 90-100% after process termination (AMD Adrenalin known issue RN-RAD-WIN-22-11-1). 3. Log warning recommending TdrDelay=30 on Windows iGPU systems. The default 2-second TDR timeout is too short for LLM compute — a hung dispatch triggers VK_ERROR_DEVICE_LOST and a utilization stall. Additional upstream fixes needed in llama.cpp/ggml for complete coverage: - nodes_per_submit=1 on UMA devices (PR#24872) - Device-lost recovery (destroy-recreate-reload) - vkDeviceWaitIdle before vkDestroyDevice References: ggml#21724, ggml#17297, ggml#14105, mpv#16334 Closes lemonade-sdk#2000 Co-Authored-By: Claude <noreply@anthropic.com>
…backends
FastFlowLM had a hardcoded 5-minute timeout (max_attempts=300) that
was not configurable. This prevented loading large models like
GPT-OSS 120B that need more than 5 minutes for GPU offload.
All other backends (llama.cpp, whisper, moonshine, ryzenai, sd, kokoro)
used wait_for_ready(endpoint) with the hardcoded 600s default instead
of passing timeout_seconds=0 to use the configurable global_timeout.
Changes:
- fastflowlm_server.cpp: read HttpClient::get_default_timeout() instead
of hardcoded 300 — respects config.json global_timeout (default 600s)
- llamacpp/whisper/moonshine/ryzenai/sd/kokoro server files: pass
timeout_seconds=0 to wait_for_ready() to use global_timeout
- vllm already used HttpClient::get_default_timeout() — unchanged
Users can set global_timeout in config.json or via
POST /api/v1/params {"global_timeout": 1200} for large models.
Closes lemonade-sdk#674
Co-Authored-By: Claude <noreply@anthropic.com>
- pr-agent-review.yml: DeepSeek-powered review on PR open/sync/reopen and @mention comments - qodo-merge.yml: Qodo Merge (qodo-ai/pr-agent) review on PR open/sync/reopen - Both gated on DEEPSEEK_API_KEY / QODO_API_KEY secrets being set Co-Authored-By: Claude <noreply@anthropic.com>
…nade-sdk#1003) Layer A — metrics_linux.cpp: Skip NPU sensor polling on kernels < 6.19.7. The amdxdna driver has known deadlock bugs (CVE-2026-23295) where sensor IOCTLs race with PM runtime suspend/resume, causing complete system freezes on Strix Halo. On unsafe kernels, get_npu_utilization() returns -1 immediately and logs a one-time warning advising a kernel upgrade. Layer B — model_manager.cpp + http_client: Cap download bandwidth when the total download size exceeds 50% of available system memory. On UMA architectures, download page-cache competes with NPU/GPU DMA buffers for the same RAM pool. A 70 GB model download can flood the page cache and starve the NPU driver, triggering OOM kills or system freezes. When memory is tight, downloads are throttled to ~50 MB/s via CURLOPT_MAX_RECV_SPEED_LARGE. Also adds max_recv_speed_bytes to DownloadOptions in http_client.h. Closes lemonade-sdk#1003 Co-Authored-By: Claude <noreply@anthropic.com>
…k#1792) post_stream() was applying default_timeout_seconds_ (300s/600s) when timeout_seconds=0 was passed. But the streaming call chain passes 0 meaning 'infinite timeout' (documented at wrapped_server.cpp:763). This caused long-running ROCm inference to fail at exactly the default timeout mark with 'CURL error: Timeout was reached', which the UI renders as 'Error: No content received from stream'. Fix: in post_stream(), pass 0L to CURLOPT_TIMEOUT when timeout_seconds is 0, which libcurl interprets as infinite. Callers who want a finite timeout should pass a positive value. Closes lemonade-sdk#1792 Co-Authored-By: Claude <noreply@anthropic.com>
…de-sdk#2180) Two changes: 1. Reorder RECIPE_DEFS to prefer ROCm over Vulkan on Linux for AMD GPUs. On systems where both backends are available (HSA runtime + supported GPU arch), ROCm is now preferred. This avoids the Linux GPU scheduler timeout entirely, since ROCm compute submissions are not subject to the 2s RADV/amdgpu scheduler timeout. Vulkan remains as fallback for unsupported GPUs. 2. Cap Vulkan context size at 65536 on Linux. The amdgpu kernel scheduler has a ~2s timeout per compute submission (same as Windows TDR). Long prompt processing (e.g. 262K context) produces submissions that exceed this, causing VK_ERROR_DEVICE_LOST and a crash with the RADV message 'The CS has been cancelled because the context is lost.' Closes lemonade-sdk#2180 Co-Authored-By: Claude <noreply@anthropic.com>
Extend max_loaded_models to accept a JSON object with per-device caps
alongside the existing integer format. When an object is provided, the
router enforces per-hardware limits (cpu/gpu/npu) before the per-type
LRU check, so users on heterogeneous systems like Strix Halo can express
limits such as '1 GPU model, 1 NPU model, 2 CPU models'.
Config format (backward compatible):
Integer: { "max_loaded_models": 2 }
Object: { "max_loaded_models": { "gpu": 1, "npu": 1, "total": 3 } }
Omitted device keys default to unlimited (-1).
Co-Authored-By: Claude <noreply@anthropic.com>
…sdk#2451) GLM-4.5 GGUF files embed MTP layers (nextn_predict_layers > 0), so the GGUF capability scanner sets caps.mtp=true, which causes lemonade to enable speculative decoding (--spec-type draft-mtp). However, llama.cpp's glm4-moe.cpp graph builder asserts when constructing the draft model context because the GGUF was not converted with the multimodal metadata that the graph builder expects: glm4-moe.cpp:149: This GGUF does not support multimodal. Please reconvert it. Fix: strip the 'mtp' label for models whose GGUF architecture contains 'glm4' after capability labels are applied. This is a surgical workaround until upstream llama.cpp fixes the GLM-4 MoE draft context construction. Co-Authored-By: Claude <noreply@anthropic.com>
Without these dpkg refuses to overwrite files owned by the other package when upgrading between lemonade-server and lemonade-server-minimal, which both ship /usr/local/bin/lemonade-router and other shared files. Closes lemonade-sdk#677 Co-Authored-By: Claude <noreply@anthropic.com>
Replay feat/mlx-engine-backend onto fix/rocm-best-gpu (current main). Resolved conflicts across: - CMakeLists.txt, model_types.h, backend_utils.cpp, runtime_config.cpp - recipe_options.cpp, model_manager.cpp, system_info.cpp - recipeOptions.ts, recipeOptionsConfig.ts (UI) - README.md, CI workflow, test file All conflicts resolved by keeping both moonshine and lemon-mlx entries, HEAD's pinned-saveOptions in UI config, system llamacpp backend row, and our 11 prior fixes throughout.
…name populate_model_metadata, bump to b1045-stable - forward_streaming_request now takes TelemetryCallback (6th param) - populate_static_max_context_window was renamed to populate_model_metadata - Remove stray brace from merge in runtime_config.cpp - Bump lemon-mlx backend pins: b1034-stable → b1045-stable
bong-water-water-bong
requested review from
jeremyfowers and
ramkrishna2910
as code owners
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Closed
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebase of #2010 (which replaced #1717) onto current main. Wires lemon-mlx-engine as the
lemon-mlxwrapped-server recipe.The upstream engine exposes an OpenAI-compatible HTTP API, so this stays within the existing
WrappedServerabstraction.Changes vs #2010
forward_streaming_requestupdated for newTelemetryCallbackparameterpopulate_static_max_context_window→populate_model_metadatarenameCore backend
MlxServer(src/cpp/{include,server}/lemon/backends/mlx_server.{h,cpp}) —/v1/chat/completions,/v1/completions, streaming;responses()returnsUnsupportedOperationExceptionrouter.cpp: instantiatesMlxServerforlemon-mlxrecipesmodel_types.h,runtime_config.cpp: registerslemon-mlxas a selectable backendResolved review items (from #2010 @fl0rianr, @bitgamma, Qodo)
completions_streamingTrue but no streaming/v1/completionscompletions_streaming = Falsein capabilities.pyforward_streaming_requestunsafe for/v1/completionsMlxServer::forward_streaming_requestsystem_info.cpp: Linux → x86_64 only; macOS → arm64 only*-previewpinstokenize_quoted_argsstd::istringstreamPerformance (b1045-stable, gfx1151)
Closes #1642
🤖 Generated with Claude Code