Add multi components recipes

[Xiaoyu Z (xiaoyu-work)]

Describe your changes

Add and validate multi-component optimization recipes backed by Olive and Mobius:

• Gemma 4 E2B: two independent PyTorch component builds under one shared HF output. The decoder uses asymmetric INT4 KQuant (group 32), vision uses symmetric INT4 RTN (group 128), and Olive assembles component-only shards with unchanged audio/embedding weights into a standard HF checkpoint before Mobius export.
• Qwen2.5-VL-3B-Instruct: replace three independent configs and the custom model implementation with one Olive multi-build config over Mobius decoder, vision, and embedding components.
• Qwen3-VL-2B-Instruct: export an FP32 Mobius package, dynamically quantize the decoder, convert vision/embedding internals to FP16 while preserving FP32 component interfaces, and assemble a directly loadable optimized ORT GenAI package.

The non-functional SD3 prototype was removed. Current Mobius SD3 graph construction does not match the released checkpoint architecture or pooled-conditioning contract, so retaining that recipe would be misleading.

Validation

• Gemma 4: olive run assembled the mixed-quantized HF checkpoint; olive capture-onnx-graph --use_mobius_builder exported all four components. Decoder has 205 group-32 asymmetric MatMulNBits nodes, vision has 114 group-128 symmetric nodes. Text returned The capital of France is Paris. and image inference returned A cat is shown in the image.
• Qwen2.5-VL: all three CPU INT4 component builds passed ONNX checker. Text returned The capital of France is Paris. and image inference returned Cat.
• Qwen3-VL: all three optimized components passed ONNX checker. Text returned The capital of France is Paris. and image inference returned A cat.
• CUDA-target Qwen2.5 components were built and passed ONNX checker; runtime inference was not run because this host has no CUDA execution provider.

[Copilot]

Pull request overview

Adds new multi-component Olive recipes and accompanying docs/scripts for:

• Stable Diffusion 3 Medium (diffusers): export once via Mobius, then optimize specific ONNX components independently.
• Qwen3-VL-2B-Instruct: two flows (export → per-component optimize, and GPTQ quantize decoder → export) plus an ORT GenAI inference script.

Changes:

• Introduces SD3 multi-component optimization config, end-to-end ONNX inference script, and documentation.
• Introduces Qwen3-VL multi-component configs (optimize components; GPTQ quantize-then-export), inference script, and documentation.
• Registers both as recipes via info.yml and adds local artifact .gitignore rules.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
stabilityai-stable-diffusion-3-medium-diffusers/multi_comp/sd3_optimize_components.json	Olive config to optimize SD3 exported ONNX components with per-component builds.
stabilityai-stable-diffusion-3-medium-diffusers/multi_comp/sd3_inference.py	SD3 all-ONNX inference script (tokenizers + transformer + VAE decoder).
stabilityai-stable-diffusion-3-medium-diffusers/multi_comp/README.md	Step-by-step instructions for export, per-component optimization, and inference.
stabilityai-stable-diffusion-3-medium-diffusers/multi_comp/info.yml	Registers SD3 multi-component recipe metadata.
stabilityai-stable-diffusion-3-medium-diffusers/multi_comp/.gitignore	Ignores exported/optimized outputs, cache, and generated images for SD3 flow.
stabilityai-stable-diffusion-3-medium-diffusers/LICENSE	Adds model licensing notice and links for SD3 Medium weights.
Qwen-Qwen3-VL-2B-Instruct/multi_comp/vlm_quantize_then_export.json	Olive config to GPTQ-quantize the decoder component (Torch stage) prior to export.
Qwen-Qwen3-VL-2B-Instruct/multi_comp/vlm_optimize_components.json	Olive config to optimize exported ONNX components with different pipelines per component.
Qwen-Qwen3-VL-2B-Instruct/multi_comp/vlm_inference.py	ORT GenAI inference script for text-only and image-conditioned generation.
Qwen-Qwen3-VL-2B-Instruct/multi_comp/README.md	Documents both Qwen multi-component flows and ORT GenAI inference requirements.
Qwen-Qwen3-VL-2B-Instruct/multi_comp/info.yml	Registers Qwen multi-component recipe metadata.
Qwen-Qwen3-VL-2B-Instruct/multi_comp/.gitignore	Ignores exported/optimized outputs, cache, and quantized HF checkpoint for Qwen flow.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.

Suppressed comments (6)

stabilityai-stable-diffusion-3-medium-diffusers/multi_comp/sd3_inference.py:20

• The documented export and optimization flow creates exported_pkg, but the inference default and its recovery command use exported_sd3_full2 (and the README repeats that default). As a result, the documented inference command fails immediately after the documented export. Align the constant, recovery command, and README on exported_pkg.

ONNX_DIR = "exported_sd3_full2"

stabilityai-stable-diffusion-3-medium-diffusers/multi_comp/sd3_inference.py:63

• The same CLIP output-contract issue occurs for CLIP-G: SD3 takes pooled projection from output 0 and conditioning from the penultimate hidden state, not output 0 as hidden and output 1 as pooled. Read the exported outputs by name and ensure the penultimate hidden state is exported.

    clip_g_hidden = out_g[0]  # last_hidden_state [1, 77, 1280]
    clip_g_pooled = out_g[1]  # text_embeds [1, 1280]

Qwen-Qwen3-VL-2B-Instruct/multi_comp/README.md:138

• This path disagrees with the recipe's output_dir: "vlm_decoder_gptq_hf", the following export command, and .gitignore; Olive writes the checkpoint at the working-directory root, not under out/. Following this displayed path sends users to a directory that is never created.

out/vlm_decoder_gptq_hf/

Qwen-Qwen3-VL-2B-Instruct/multi_comp/README.md:115

• This URL points to Microsoft's unrelated Apache Spark Mobius repository. The ONNX model builder referenced by Olive is onnxruntime/mobius, so this link does not provide the promised ORT GenAI examples.

For the `genai_config.json` structure, see the
[Mobius ORT GenAI examples](https://github.com/microsoft/mobius/tree/main/examples) which write the
config automatically.

Qwen-Qwen3-VL-2B-Instruct/multi_comp/vlm_inference.py:55

• Passing the raw text alongside images does not insert Qwen's image placeholder tokens. The embedding graph uses those tokens to splice the vision features into the sequence, so this path can ignore the image or fail on incompatible inputs. Build an image/text chat message and apply the tokenizer's chat template before calling the processor, as builtin/inference.py:53-87 does.

    images = og.Images.open(image_path)
    inputs = processor(prompt, images=images)

Qwen-Qwen3-VL-2B-Instruct/multi_comp/README.md:99

• MobiusBuilder writes the vision sidecar as image_processor.json, not vision_processor.json. Since no step creates the listed filename, the setup layout is inaccurate and users cannot match it to the generated GenAI config.

This issue also appears on line 113 of the same file.

  vision_processor.json      # Vision preprocessing config

[Xiaoyu Z (xiaoyu-work)]

Copilot resolve the merge conflicts in this pull request

[Copilot]

Copilot resolve the merge conflicts in this pull request

Resolved the merge conflicts by merging origin/main into this PR branch in merge commit 84db78b. I also fixed the previously failing missing final newline in the Qwen3-VL multi-comp .gitignore while resolving the merge.

[Jambay Kinley (jambayk)]

is it better to quantize embedding separately from the decoder/lm_head? for example when the embedding and lm head share weights and we use the same algorithm and quant config for both, we would ideally like to tie their weights together during quantization as well and export

[Jambay Kinley (jambayk)]

we also have an "embed" option in the quantize passes and i think for rtn atleast it knows whether to keep the lm head and embeds tied if both are set to true.

[Xiaoyu Z (xiaoyu-work)]

good call. I'll update Mobius to provide the info about shared weights then Olive can do quantization once.