# Gen AI Builder Overview ## What is the Gen AI Builder? The Gen AI Builder is a Python API that automates step 2 of the typical three-step LLM deployment workflow (quantize → **compile & package** → deploy). It takes a quantized ONNX model (produced by step 1 of the [QNN model preparation notebooks](https://qpm.qualcomm.com/#/main/tools/details/Tutorial_for_Llama3)) and produces a `GenAIContainer` ready for on-device inference with a single `build()` call. NOTEBOOK / CLI PIPELINE (manual) ============================================================ AR/CL Split MHA2SHA Convert Quantize LoRA Context Convert --> ONNX --> Transform --> to DLC --> DLC --> Import --> Binary Gen 7 stages x (AR x CL x splits) = hundreds of CLI invocations GEN AI BUILDER API (automated) ============================================================ Factory Configure builder.build() .create() --> set_targets --> +---------------------------+ native_kv | All 7 stages automated | Auto-detects multi_graph | in a single call | model arch lora_config +---------------------------+ | v GenAIContainer (ready for deployment) 3 API calls replace the entire notebook Copy to clipboard ### Supported Models The factory auto-detects model architecture from `config.json`. Preconfigured builders exist for all architectures listed in [`SupportedLLMs`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-gen-ai-api-gen-ai-builder-factory.html#qairt.gen_ai_api.gen_ai_builder_factory.SupportedLLMs), including Llama, Qwen, Phi, Mistral, Baichuan, and others. Unsupported architectures fall back to a default `GenAIBuilderHTP` with a warning. Note See [Appendix](https://docs.qualcomm.com/doc/80-87189-2/topic/appendix.html#appendix) for verified model/platform combinations and architecture-specific notes. ### Large Multimodal Models (LMM) Vision-language models such as **Qwen2.5-VL** and **Qwen3-VL** consist of two cooperating components: an image encoder (Vision Transformer) and a text decoder (causal LLM). The Gen AI Builder handles each component independently and then wires them together using the `WorkflowBuilder` API. VISION-LANGUAGE PIPELINE (LMM) ============================================================ image encoder text decoder ONNX (ViT) ─────────────────────────────► ONNX (LLM) │ │ ▼ ▼ VisionEncoderBuilderHTP GenAIBuilderHTP (no AR/CL, no splitting) (full AR/CL + split pipeline) │ │ └──────────────── WorkflowBuilder ──────────┘ │ ▼ WorkflowContainer (both components bundled) │ ▼ ImageT2TExecutor (image + text → generated text) Copy to clipboard The factory automatically selects the correct architecture-specific subclass for each component. For Qwen2.5-VL this is `Qwen2VLVisionEncoderBuilderHTP` and for Qwen3-VL it is `Qwen3VLVisionEncoderBuilderHTP`, which is selected by reading the `architectures` field from `config.json`. **Key differences from a single-model LLM build:** - `GenAIBuilderFactory.create_vision_encoder()` builds the image encoder with a simplified pipeline (no AR/CL conversion or ONNX splitting). - `GenAIBuilderFactory.create()` builds the text decoder as normal. - `WorkflowBuilder.from_builders()` combines both builders and produces a `WorkflowContainer` that bundles the sub-containers for on-device execution. - Execution uses `ImageT2TExecutor` (returned by `container.get_executor(device)`) and accepts a `GenerationRequest` with interleaved image and text content. See [Vision-Language Model Inference on HTP](https://docs.qualcomm.com/doc/80-87189-2/topic/lmm_builder.html#lmm-builder) for the step-by-step tutorial. See [WorkflowBuilder Guide](https://docs.qualcomm.com/doc/80-87189-2/topic/workflow_builder.html#workflow-builder) for the full WorkflowBuilder API reference and topology patterns. ### What the Builder Does Automatically Note The builder expects a quantized ONNX model and its associated encodings file as input. These are produced by step 1 of the QNN model preparation notebooks (the AIMET quantization step). See the [notebook tutorials](https://qpm.qualcomm.com/#/main/tools/details/Tutorial_for_Llama3) if you have not yet quantized your model. **Single-component LLM** When you call `build()` on a `GenAIBuilderHTP`, the builder automates the following stages that are manual in notebooks: 1. **AR/CL conversion** – generates ONNX models for each AR x CL combination 2. **ONNX splitting** – partitions the model into N splits 3. **MHA2SHA transformation** – converts multi-head to single-head attention per split 4. **ONNX to DLC conversion** – with quantization overrides from encodings 5. **DLC quantization** – `act_bitwidth=16`, `bias_bitwidth=32` 6. **LoRA graph building and import** – when `lora_config` is set 7. **Context binary generation** – with weight sharing and native KV format config **Vision-language model (LMM)** For LMMs the two components go through different pipelines: - **Image encoder** (`VisionEncoderBuilderHTP`): a simplified single-pass pipeline with no AR/CL conversion and no ONNX splitting. The ViT ONNX is converted to DLC, quantized, and compiled directly into a context binary. - **Text decoder** (`GenAIBuilderHTP`): the full 7-stage pipeline above, identical to a standalone LLM build. `WorkflowBuilder.build()` runs both pipelines and bundles the resulting containers into a single `WorkflowContainer`. The two builds are independent and cache separately, so changing the text decoder configuration does not invalidate the image encoder cache and vice versa. ### Understanding the Build Cache The builder uses a content-addressed cache to avoid redundant work across runs. **Single-component LLM**: all build stages (`arcl`, `transform`, `convert`, `compile`) share a single `cache_root`. Changing any input that affects a stage’s hash invalidates that stage and everything downstream; earlier stages are reused. **Vision-language model (LMM)**: the image encoder and text decoder maintain independent cache trees under the same `cache_root`. Each component is keyed on its own ONNX artifact and configuration, so updating the text decoder (for example, changing the number of splits or the target chipset) does not invalidate the image encoder cache and vice versa. This means a typical iterative workflow (where the ViT is stable and only the LLM configuration is being tuned) pays the full build cost for the text decoder but hits the cache for the image encoder on every run. Cache keys are SHA-256 hashes of the input configuration and model content. Each stage stores the intermediate artifacts for that exact configuration in its own subdirectory. On subsequent runs, any stage whose inputs are unchanged is skipped. Changing any option — targets, split count, AR numbers, context lengths, etc. — produces a new hash and triggers a full rebuild of affected stages. ## Cache directory layout Artifacts are organized two levels deep: a human-readable top-level directory per model (slugified from the model file name), then one subdirectory per build stage named `_`, where `` is one of `arcl`, `transform`, `convert`, or `compile` and `` is the first 32 hex characters of the stage’s SHA-256 content hash. The following example shows the layout for a `llama32` model compiled into 3 splits with AR 128 and context length 4096: / └── llama32/ # human-readable model group (slugified model stem) ├── arcl_14a468361bda46c6ab0338f1b6.../ # AR/CL converted ONNX (when enabled) │ ├── ar1_cl4096.onnx │ ├── ar128_cl4096.onnx │ └── builder_cache_info.json ├── transform_19d471ebbd72475187007c.../ # per-split transformed ONNX (one subdir per split) │ ├── ar128_cl4096_3_of_3/ │ │ ├── ar128_cl4096_3_of_3.onnx │ │ ├── ar128_cl4096_3_of_3.data │ │ └── ar128_cl4096_3_of_3.encodings │ ├── ... │ └── builder_cache_info.json ├── convert_cc7d4fcc8533e37b374e1ae3.../ # converted DLC for a split │ ├── ar128_cl4096_3_of_3.dlc │ └── builder_cache_info.json └── compile_1ce356f11209360ab98a88c1.../ # context binary for a split ├── ar1_cl4096_3_of_3.bin ├── ar1_cl4096_3_of_3_cache_info.json # QNN context binary metadata └── builder_cache_info.json Copy to clipboard Each stage directory contains a `builder_cache_info.json` sidecar for offline inspection of what produced that cache entry. It has the following fields: | Field | Description | | --- | --- | | `operation` | The build stage: `arcl`, `transform`, `convert`, or `compile`. | | `source` | Path to the input artifact that was hashed for this stage. | | `config` | The configuration options that were hashed, as a dictionary of stringified values (for inspection only — see the note below). | | `hash` | The full 64-character SHA-256 hash (the directory name uses the first 32 characters). | | `created_at` | ISO-8601 UTC timestamp of when the entry was written. | For example, a `convert` stage sidecar records the converter options used to produce the DLC: { "operation": "convert", "source": ".../transform_19d471.../ar128_cl4096_3_of_3/ar128_cl4096_3_of_3.onnx", "config": { "export_format": "DLC_DEFAULT", "act_precision": "16", "weights_precision": "8", "per_channel_quantization": "True" }, "hash": "cc7d4fcc8533e37b374e1ae38423b42e741168d613bb3602137466ae4f63ed2f", "created_at": "2026-06-10T04:47:10.673060+00:00" } Copy to clipboard Note The `config` values are stringified for human inspection; they are **not** a serialized object you can load directly back into a config class. To reuse settings programmatically, read the values you need and reconstruct the corresponding config object (see [Replaying a stage from the cache](https://docs.qualcomm.com/doc/80-87189-2/topic/genai_overview.html#genai-builder-cache-recompile)). ## Invalidating the cache To force a full rebuild (for example, after updating the SDK): - Delete the stage directory (or the whole model group) for the affected configuration, or - Point `cache_root` to a new location, or - Modify any input that changes the content hash (model file, config options, etc.) ## Inspecting a partial build When a build fails mid-way, the cache directory will contain whatever artifacts were completed before the failure. To see which splits compiled successfully: find -name '*.bin' Copy to clipboard ## Replaying a stage from the cache Each stage’s `builder_cache_info.json` records the full configuration used to produce its output, so any stage can be replayed independently without rerunning the whole builder. The example below shows re-running the compile stage for a single converted split using the recorded settings. The same approach applies to convert, transform, and arcl stages. import json import qairt from qairt.api.compiler.config import CompileConfig cache_root = "/llama32" # 1. Inspect the compile settings recorded for a previous build. with open(f"{cache_root}/compile_/builder_cache_info.json") as f: compile_info = json.load(f) print(compile_info["config"]["backend"]) # e.g. "BackendType.HTP" print(compile_info["config"]["soc_details"]) # e.g. "chipset:SM8850" # 2. Load a converted split (the .dlc produced by a convert stage). split = qairt.load(f"{cache_root}/convert_/my_split.dlc") # 3. Reconstruct a CompileConfig from the inspected settings and re-compile. config = CompileConfig(backend="HTP", soc_details="chipset:SM8850") compiled = qairt.compile(split, config=config) compiled.save("my_split_recompiled.bin") Copy to clipboard The `config` dictionary in the sidecar lists every option that was applied. To reload the full configuration as a pydantic object rather than constructing it by hand, pass the dict directly to the config class. The sidecar may contain extra keys; set `extra="allow"` on the model config before calling `model_validate`: CompileConfig.model_config["extra"] = "allow" config = CompileConfig.model_validate(compile_info["config"]) Copy to clipboard Last Published: Sep 17, 2026 [Previous Topic Gen AI Builder](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/guides.md) [Next Topic Configuring the Gen AI Builder](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/genai_builder_configuration.md)