Pipeline block ④ (Host: weights in the registry). Part of the compile-to-weights epic.
Context. ModelRegistry already gives content-hash pinning (verified at load), version-skew gating, and production/staging aliases — exactly the "recompile cycle as CI/CD" the paper wants — but only for sklearn/xgboost/onnx. It has no LLM-weights runtime, so a compiled Qwen checkpoint can't be pinned/served/rolled-back with the same guarantees. Relates to #56 (MLNode runtime extension), #41 (node entry-point groups).
Proposal. Add an LLM-weights runtime.
- Extend
_RUNTIMES (registry.py:42) + the SQL CHECK (registry.py:146) with 'vllm'/'transformers'.
- New serve helper in
ml/loaders.py analogous to get_onnx_session (:247) + a pool mirroring _ONNX_SESSION_CACHE (:51), keyed by (model_id, version) — lazily connect to (or stand up) a vLLM OpenAI-compatible endpoint.
- Extend
nodes/ml.py Runtime Literal (:27) + _load/_predict dispatch.
- Relax the
file:// single-file assumption (_verify_content_hash registry.py:347, _file_uri_to_path loaders.py:60) for a directory / HF-repo artifact; content hash over a multi-GB weights dir needs streaming/merkle, not whole-file read_bytes (registry.py:350).
Acceptance criteria.
Caveat. "Stargraph owns the GPU server lifecycle" is XL; connect-to-external-vLLM is the cheap, recommended first form.
Source: arXiv:2605.22502v1 — "Compiling Agentic Workflows into LLM Weights" (subterranean agents). Part of the end-to-end compile-to-weights pipeline epic; file refs verified against current main. Determinism: per-graph knob (fully-compiled | hybrid | rules-only) defaults to honor #117 (no learned core router).
Pipeline block ④ (Host: weights in the registry). Part of the compile-to-weights epic.
Context.
ModelRegistryalready gives content-hash pinning (verified at load), version-skew gating, and production/staging aliases — exactly the "recompile cycle as CI/CD" the paper wants — but only forsklearn/xgboost/onnx. It has no LLM-weights runtime, so a compiled Qwen checkpoint can't be pinned/served/rolled-back with the same guarantees. Relates to #56 (MLNode runtime extension), #41 (node entry-point groups).Proposal. Add an LLM-weights runtime.
_RUNTIMES(registry.py:42) + the SQL CHECK (registry.py:146) with'vllm'/'transformers'.ml/loaders.pyanalogous toget_onnx_session(:247) + a pool mirroring_ONNX_SESSION_CACHE(:51), keyed by(model_id, version)— lazily connect to (or stand up) a vLLM OpenAI-compatible endpoint.nodes/ml.pyRuntime Literal (:27) +_load/_predictdispatch.file://single-file assumption (_verify_content_hashregistry.py:347,_file_uri_to_pathloaders.py:60) for a directory / HF-repo artifact; content hash over a multi-GB weights dir needs streaming/merkle, not whole-fileread_bytes(registry.py:350).Acceptance criteria.
loada weights-dir artifact undervllm/transformerswith a verified streaming/merkle content hash.(model_id, version);aliasrollback works.Caveat. "Stargraph owns the GPU server lifecycle" is XL; connect-to-external-vLLM is the cheap, recommended first form.
Source: arXiv:2605.22502v1 — "Compiling Agentic Workflows into LLM Weights" (subterranean agents). Part of the end-to-end compile-to-weights pipeline epic; file refs verified against current
main. Determinism: per-graph knob (fully-compiled | hybrid | rules-only) defaults to honor #117 (no learned core router).