# WorkflowBuilder Guide

## What is the WorkflowBuilder?

A single LLM is built with [`create()`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-gen-ai-api-gen-ai-builder-factory.html#qairt.gen_ai_api.gen_ai_builder_factory.GenAIBuilderFactory.create)
and produces a `GenAIContainer`. The **WorkflowBuilder** is the next step up: it assembles
*multiple* builders into a single deployable unit — a `WorkflowContainer` — and describes how
those components connect at runtime.

The primary use case today is **Large Multimodal Models (LMMs)** such as Qwen2.5-VL and Qwen3-VL,
where a Vision Transformer (image encoder) and a causal LLM (text decoder) must cooperate. The
WorkflowBuilder handles both components with a single `build()` call and bundles the results for
on-device execution.

MULTI-COMPONENT BUILD (WorkflowBuilder)
    ============================================================
    
      VisionEncoderBuilderHTP ─┐
                                ├─► WorkflowBuilder.build() ─► WorkflowContainer
      GenAIBuilderHTP          ─┘                                      │
                                                                        ▼
                                                              WorkflowContainer.save()
                                                              WorkflowContainer.load()
                                                                        │
                                                                        ▼
                                                             container.get_executor(device)
                                                                        │
                                                                        ▼
                                                             ImageT2TExecutor  /  T2TExecutor
    Copy to clipboard

### When to use the WorkflowBuilder

Use `WorkflowBuilder` when:

- You are building a vision-language model (LMM) with separate encoder and decoder artifacts.
- Your model has any multi-component topology with directed data flow between components.

Use [`create()`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-gen-ai-api-gen-ai-builder-factory.html#qairt.gen_ai_api.gen_ai_builder_factory.GenAIBuilderFactory.create) directly when:

- You are building a single-component text-only LLM.

Note

`WorkflowBuilder.from_builders()` accepts a single-entry `builders` dict with no
`workflow_graph` argument as a convenience wrapper for the single-component case. This
is equivalent to calling `GenAIBuilderFactory.create()` and produces a `T2TExecutor`
when executed.

## Core Concepts

### WorkflowGraph

A `WorkflowGraph` is a directed graph that describes the topology of a multi-component model.
It is defined by an ordered tuple of nodes and an optional tuple of directed connections between
them.

from qairt.gen_ai_api.configs.workflow import WorkflowGraph, WorkflowNode, WorkflowNodeRole
    
    graph = WorkflowGraph(
        nodes=(
            WorkflowNode(name="vision", role=WorkflowNodeRole.IMAGE_ENCODER),
            WorkflowNode(name="text",   role=WorkflowNodeRole.TEXT_GENERATOR,
                         output_callback_type="textCallback"),
        ),
        connections=(
            ("vision", "text", "standard"),
        ),
    )
    Copy to clipboard

### Graph Validation Rules

`WorkflowGraph` is **frozen** (Pydantic `ConfigDict(frozen=True)`): field reassignment is
not allowed after construction. Any graph change requires constructing a new `WorkflowGraph`.
This ensures the validation below always runs against the final state of the data.

Four rules are checked at construction time and raise `ValueError` immediately:

| Rule | Error message |
| --- | --- |
| No duplicate node names | `"Duplicate node name(s) in workflow graph: ['name', ...]"` |
| Each connection must be exactly a 3-tuple | `"Connection at index N must have exactly 3 elements [source, target, connection_type], got M: ..."` |
| Connection endpoints must reference declared nodes | `"Connection at index N references node(s) not declared in the graph: ['name']. Declared nodes: [...]"` |
| `connection_type` must be `"standard"` or `"wildcard"` | `"Connection at index N has invalid connection_type '...'. Must be one of: ['standard', 'wildcard']"` |

### WorkflowNode

Each node in the graph represents one model component. The fields are:

| Field | Required | Description |
| --- | --- | --- |
| `name` | Yes | Unique identifier for the node within this graph. Used as the key in `builders` and<br>`containers` dictionaries. |
| `role` | Yes | Functional role of the component (see [Node Roles](https://docs.qualcomm.com/doc/80-87189-2/topic/workflow_builder.html#node-roles) below). |
| `output_callback_type` | No | Genie callback type string for streaming output from this node. Typically<br>`"textCallback"` for the text decoder. |
| `path` | Conditional | Path to model artifacts. Required when using `WorkflowBuilder.from_workflow()`;<br>unused by `WorkflowBuilder.from_builders()`. |
| `tokenizer_path` | No | Explicit path to the tokenizer. Overrides auto-discovery from `path`. |
| `config_path` | No | Explicit path to the model `config.json`. Overrides auto-discovery from `path`. |

### Node Roles

The `WorkflowNodeRole` enum defines the functional role of each component in the graph. The
role determines which executor is selected by `WorkflowContainer.get_executor()` and how the
Genie pipeline wires the components together at runtime.

| Role | Description |
| --- | --- |
| `TEXT_GENERATOR` | Causal language model that produces text output. Every workflow must have exactly<br>one `TEXT_GENERATOR` node. |
| `IMAGE_ENCODER` | Vision Transformer that processes image input into feature embeddings fed to the<br>text decoder. Presence of this role triggers `ImageT2TExecutor`. |
| `TEXT_ENCODER` | Encodes text into embeddings for injection into the text decoder. The builder<br>automatically injects a `TEXT_ENCODER` node when `embedding_config` is present. |

### Connections and Connection Types

Connections are directed edges in the graph. Each connection is a three-tuple:

connections=(
        (source_node_name, target_node_name, connection_type),
    )
    Copy to clipboard

Two connection types are supported:

| Type | Behaviour |
| --- | --- |
| `"standard"` | Role-based connector mapping. Genie automatically selects the connector based on<br>the source and target node roles. Use this for all standard topologies (e.g. image<br>encoder → text generator). |
| `"wildcard"` | Bypasses role-based connector routing. Required for Qwen3-VL and similar models<br>where the vision encoder output does not map to a fixed role-based connector. |

## Building a Workflow

There are two factory methods. Choose based on whether you have already constructed the
component builders.

### from\_builders

Use `from_builders` when you have fully configured `GenAIBuilder` (or
`VisionEncoderBuilderHTP`) instances and want to wire them into a workflow.

from qairt.gen_ai_api.builders.workflow_builder import WorkflowBuilder
    
    workflow_builder = WorkflowBuilder.from_builders(
        builders={"vision": vision_builder, "text": text_builder},
        workflow_graph=graph,
    )
    container = workflow_builder.build()
    Copy to clipboard

Parameters:

- **builders** – `Dict[str, Buildable]`. Keys must match the `name` fields of the nodes
in `workflow_graph`.
- **workflow\_graph** – `Optional[WorkflowGraph]`. If omitted and `builders` has exactly
one entry, a single-node `TEXT_GENERATOR` graph is created automatically. If omitted and
`builders` has more than one entry, a `ValueError` is raised.

Raises:

- `ValueError` – if `workflow_graph` is `None` and `builders` has more than one entry.
- `ValueError` – if `workflow_graph` is `None` and `builders` is empty.

### from\_workflow

Use `from_workflow` when you have a `WorkflowGraph` whose nodes each carry a `path` and
want the builder to construct the component builders automatically from those paths.

from qairt.gen_ai_api.builders.workflow_builder import WorkflowBuilder
    from qairt.gen_ai_api.configs.workflow import WorkflowGraph, WorkflowNode, WorkflowNodeRole
    
    graph_with_paths = WorkflowGraph(
        nodes=(
            WorkflowNode(
                name="vision",
                role=WorkflowNodeRole.IMAGE_ENCODER,
                path="/path/to/vit.onnx",
                config_path="/path/to/config.json",
            ),
            WorkflowNode(
                name="text",
                role=WorkflowNodeRole.TEXT_GENERATOR,
                path="/path/to/llm.onnx",
                tokenizer_path="/path/to/tokenizer.json",
            ),
        ),
        connections=(("vision", "text", "standard"),),
    )
    
    workflow_builder = WorkflowBuilder.from_workflow(
        workflow=graph_with_paths,
        backend_type=BackendType.HTP,
        cache_root="./cache",
    )
    container = workflow_builder.build()
    Copy to clipboard

Parameters:

- **workflow** – `WorkflowGraph`. Every node must have a `path` set.
- **backend\_type** – `BackendType`. Backend for all component builders. Defaults to
`BackendType.HTP`.
- **cache\_root** – `Optional[os.PathLike]`. Shared cache root applied to every component
builder. Defaults to `None`.

Raises:

- `ValueError` – if any node is missing a `path`.

## WorkflowContainer Lifecycle

### Building

Calling `build()` on the `WorkflowBuilder` invokes `build()` on each component builder
and assembles the resulting containers into a `WorkflowContainer`:

container = workflow_builder.build()
    # container is a WorkflowContainer
    Copy to clipboard

Note

If the built `TEXT_GENERATOR` container uses a split embedding table, the builder
automatically inserts a `TEXT_ENCODER` node named `<text_generator_name>_encoder`
(e.g. `"text_encoder"` for a node named `"text"`) into the returned
`WorkflowContainer`.

### Saving and Loading

`WorkflowContainer` serialises all sub-containers and the workflow graph to a single
directory. Load the directory back to get a `WorkflowContainer` ready for execution without
rebuilding.

# Save after build
    container.save("./my_workflow_container", exist_ok=True)
    
    # Load in a later session (no rebuild required)
    from qairt.gen_ai_api.containers.workflow_container import WorkflowContainer
    container = WorkflowContainer.load("./my_workflow_container")
    Copy to clipboard

`save()` parameters:

- **dest** – destination directory path.
- **exist\_ok** – if `False` (default), raises an error if the directory already exists.

`load()` raises:

- `NotADirectoryError` – if `path` is not a directory.
- `FileNotFoundError` – if `metadata.json` is missing from the directory.

### Getting an Executor

Call `get_executor()` on a loaded (or freshly built) `WorkflowContainer` to obtain an
executor suitable for on-device inference. The method inspects the workflow topology and
returns the appropriate executor type automatically:

| Workflow topology | Executor returned |
| --- | --- |
| `TEXT_GENERATOR` only, or `TEXT_ENCODER` → `TEXT_GENERATOR` | `T2TExecutor` |
| Any topology containing `IMAGE_ENCODER` | `ImageT2TExecutor` |

executor = container.get_executor(
        device,
        engine_config=None,   # optional EngineConfig
        clean_up=True,        # delete on-device artifacts after execution
    )
    Copy to clipboard

Parameters:

- **device** – optional `Device` instance. When omitted, the SDK picks the connected device.
- **engine\_config** – optional `EngineConfig` for runtime engine settings.
- **\*\*kwargs** – forwarded to the concrete executor constructor (e.g. `qairt_sdk_root`,
`clean_up`).

Raises:

- `ValueError` – if the workflow contains no nodes.
- `NotImplementedError` – if no `TEXT_GENERATOR` node is found.
- `NotImplementedError` – if the topology is not yet supported.

### Running Inference

After obtaining an executor, call `generate()` with a `GenerationRequest`:

from qairt.gen_ai_api.configs.generation_config import GenerationRequest
    
    # Text-only
    result = executor.generate(GenerationRequest(messages="Hello!"))
    
    # Multimodal (image + text) — ImageT2TExecutor only
    result = executor.generate(
        GenerationRequest(
            messages=[
                {
                    "role": "user",
                    "content": [
                        {"type": "image", "image": "/path/to/image.raw"},
                        {"type": "text",  "text": "Describe this image."},
                    ],
                }
            ]
        )
    )
    result.print()
    Copy to clipboard

For `ImageT2TExecutor`, image paths must point to pre-processed `.raw` files in the
format expected by the model’s vision encoder. See [Vision-Language Model Inference on HTP](https://docs.qualcomm.com/doc/80-87189-2/topic/lmm_builder.html#lmm-builder) for the image
preprocessing steps.

After inference is complete, release on-device resources:

executor.clean_environment()
    Copy to clipboard

## Workflow Patterns

### Vision-Language Model (standard connection)

The standard topology for Qwen2.5-VL: an image encoder feeds into the text generator over
a `"standard"` connection.

graph = WorkflowGraph(
        nodes=(
            WorkflowNode(name="vision", role=WorkflowNodeRole.IMAGE_ENCODER),
            WorkflowNode(name="text",   role=WorkflowNodeRole.TEXT_GENERATOR,
                         output_callback_type="textCallback"),
        ),
        connections=(("vision", "text", "standard"),),
    )
    
    container = WorkflowBuilder.from_builders(
        builders={"vision": vision_builder, "text": text_builder},
        workflow_graph=graph,
    ).build()
    Copy to clipboard

### Vision-Language Model (wildcard connection)

Some models (e.g. Qwen3-VL) require a `"wildcard"` connection because the vision encoder
output does not map to a fixed role-based connector. The factory sets this automatically when
it detects the architecture; if you are constructing the graph manually, use:

graph = WorkflowGraph(
        nodes=(
            WorkflowNode(name="vision", role=WorkflowNodeRole.IMAGE_ENCODER),
            WorkflowNode(name="text",   role=WorkflowNodeRole.TEXT_GENERATOR,
                         output_callback_type="textCallback"),
        ),
        connections=(("vision", "text", "wildcard"),),
    )
    Copy to clipboard

### Single-Component Text Generator

For completeness, `WorkflowBuilder` also supports wrapping a single text-only builder.
Omit the `workflow_graph` argument and a `TEXT_GENERATOR` graph is created automatically:

container = WorkflowBuilder.from_builders({"text": genai_builder}).build()
    executor = container.get_executor(device)  # returns T2TExecutor
    Copy to clipboard

For a complete end-to-end example that builds a vision-language model from ONNX artifacts,
compiles for a target chipset, and runs multimodal inference on device, see [Vision-Language Model Inference on HTP](https://docs.qualcomm.com/doc/80-87189-2/topic/lmm_builder.html#lmm-builder).

## API Reference

The full auto-generated API documentation for all workflow classes is in the API section:

- [WorkflowBuilder, VisionEncoderBuilderHTP](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-gen-ai-api-builders.html)
- [WorkflowContainer](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-gen-ai-api-containers.html)
- [WorkflowGraph, WorkflowNode, WorkflowNodeRole](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-gen-ai-api-configs-workflow.html)

Last Published: Aug 26, 2026

[Previous Topic
Backend Extension JSON Files](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/genai_migration.md) [Next Topic
ONNX Optimizer](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/guides.md)