# LLM Pipeline Stages Built-in stages for the LLM pipeline. Each stage implements the three-class contract (Input, Config, Output) and is registered in the global stage registry. ## ModelLoadingStage Model Loading Stage This stage handles loading and re-authoring models using QcAutoModelForCausalLM. - *class* qairt.experimental.pipeline.torch.llm.stages.model\_loading\_stage.ExportFormat(*value*) - Bases: `str`, `Enum` Supported export formats for `ModelLoadingStage.export()`. - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.model\_loading\_stage.ModelLoadingConfig - Bases: [`StageConfig`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageConfig) Configuration for model loading stage. - hf\_config - Optional HuggingFace config to pass to QcAutoModelForCausalLM. - model\_config\_overrides - Optional overrides for model config (HF config + QC config) after loading. - hf\_pretrained\_kwargs - kwargs forwarded to QcAutoModelForCausalLM.from\_pretrained (e.g. trust\_remote\_code, revision, etc.). - model\_reauthoring - Whether to re-author the model via QcAutoModelForCausalLM. - qc\_config - Populated by \_pre\_hook when model\_reauthoring=True. - load\_tokenizer - Whether to load the tokenizer. - hf\_tokenizer\_kwargs - Forwarded to AutoTokenizer.from\_pretrained. - apply\_default\_adaptations - Whether to apply default backend adaptations. - custom\_reauthoring\_artifacts - Directory with mappings.py + reauthoring.py; requires model\_reauthoring=True. - Validators - - `_check_custom_reauthoring_requires_reauthoring` » `all fields` - *field* apply\_default\_adaptations*: bool* *= True* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - *field* custom\_reauthoring\_artifacts*: Optional[Path]* *= None* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - *field* hf\_config*: PretrainedConfig | None* *= None* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - *field* hf\_pretrained\_kwargs*: Dict[str, Any]* *[Optional]* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - *field* hf\_tokenizer\_kwargs*: Dict[str, Any]* *[Optional]* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - *field* load\_tokenizer*: bool* *= True* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - *field* model\_config\_overrides*: Dict[str, Any]* *[Optional]* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - *field* model\_reauthoring*: bool* *= True* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - *field* qc\_config*: Optional[Any]* *= None* - - Validated by - - `_check_custom_reauthoring_requires_reauthoring` - model\_post\_init(*context: Any*, */*) → None - We need to both initialize private attributes and call the user-defined model\_post\_init method. - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.model\_loading\_stage.ModelLoadingOutput - Bases: [`StageOutput`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageOutput) Output from model loading stage. - *field* config*: Optional[Any]* *= None* - - *field* fp\_model*: Optional[torch.nn.Module]* *= None* - - *field* is\_reauthored*: bool* *= False* - - *field* lora\_metadata*: Optional['LoRAAdapterMetadata']* *= None* - - *field* model*: torch.nn.Module* *[Required]* - - *field* tokenizer*: Any | None* *= None* - - *class* qairt.experimental.pipeline.torch.llm.stages.model\_loading\_stage.ModelLoadingStage - Bases: [`Stage`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.Stage) Stage for loading and re-authoring models using QcAutoModelForCausalLM. - Config - alias of `ModelLoadingConfig` - Output - alias of `ModelLoadingOutput` - evaluate(*stage\_output: ModelLoadingOutput*, *config: ModelLoadingConfig*, *\*\*kwargs: Any*) → Dict[str, float] - Run the configured evaluation metrics on the loaded model. - Parameters - - **stage\_output** – Output from this stage, providing the model and tokenizer. - **config** – Stage configuration carrying `evaluator_config` and, via the pipeline context, the `cache_dir` default output location. - **\*\*kwargs** – Only `evaluator_config` is accepted, to override the one on the stage/pipeline config; it takes precedence when provided. Any other keyword argument raises `TypeError` (guards against silent typos). - Returns - The `{display_name: score}` mapping from `run_evaluation()`. - Raises - - **ValueError** – If no `evaluator_config` with metrics is configured, if the tokenizer was not loaded, or if a reauthored model is evaluated without a `generator_config` (required for sequence/context lengths). - **TypeError** – If an unexpected keyword argument is passed. - *classmethod* export(*stage\_output: ModelLoadingOutput*, *output\_path: Path*, *format: ExportFormat = ExportFormat.HF*, *\*\*kwargs: Any*) → None - Export the loaded model in the requested format. - Parameters - - **stage\_output** – Output containing the loaded model. - **output\_path** – Destination directory (HF) or file path (ONNX). - **format** – `HF` (default) or `ONNX`. - **\*\*kwargs** – Forwarded to the underlying export function. - generate(*stage\_output: ModelLoadingOutput*, *config: ModelLoadingConfig*, *prompt: Union[str, list[dict[str, str]], Path]*, *\*\*kwargs: Any*) → Any - Optional generation method using the loaded model. - Parameters - - **stage\_output** – Output from the model loading stage. - **config** – Stage configuration. - **prompt** – Input text to generate from. Can be a string, a Path to a text file, or a list of chat messages (in which case the last message content is used). - **\*\*kwargs** – Additional keyword arguments forwarded to the underlying generate. - *classmethod* load\_from\_cache(*cache\_path: Path*, *\*\*kwargs: Any*) → ModelLoadingOutput - Restore a previously exported model from `cache_path`. Loads the fully adapted model (including ConvInplaceLinear) directly from the torch-saved artifact, then re-applies the KV cache class-level patches for re-authored models (these are not preserved in pickle). - Keyword Arguments - **lora\_cfg** – Optional LoRAFeatureConfig to recompute adapter metadata (not persisted to cache). - qairt.experimental.pipeline.torch.llm.stages.model\_loading\_stage.load\_lora\_adapter\_metadata(*lora\_cfg: [LoRAFeatureConfig](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-common.html#qairt.experimental.pipeline.torch.common.configs.LoRAFeatureConfig)*) → [LoRAAdapterMetadata](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-common.html#qairt.experimental.pipeline.torch.common.configs.LoRAAdapterMetadata) - Load and validate LoRA adapter metadata, compute alpha scalings. - Parameters - **lora\_cfg** – LoRA feature configuration from the pipeline recipe. - Returns - LoRAAdapterMetadata containing peft configs, alpha scalings, and max scale. - Raises - - **FileNotFoundError** – If any adapter path does not exist. - **ValueError** – If PeftConfig is missing r or lora\_alpha fields. - **RuntimeError** – If PeftConfig.from\_pretrained fails for any adapter. ## QuantizationStage Quantization Stage Runs a named AIMET quantization recipe against a loaded model and produces a quantized model artifact (QuantizationSimModel + encodings). - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.quantization\_stage.QuantizationConfig - Bases: [`StageConfig`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageConfig) Configuration for the quantization stage. - recipe\_name - Registry key for the recipe to run (e.g. `"lpbq_seqmse"`). - recipe\_config - Recipe configuration — either a path to a YAML file or an inline dict. `None` falls back to the recipe class default config. - technique\_kwargs - Per-technique parameter overrides forwarded verbatim as top-level config overrides. Use nested dicts keyed by technique name, e.g. `{"seqmse": {"num_batches": 10}, "lpbq": {"decompressed_bw": 4}}`. - config\_overrides - Arbitrary top-level config overrides applied on top of `recipe_config` before technique\_kwargs. - filename\_prefix - Prefix for exported ONNX/encodings filenames (e.g. `"Llama-3.2-1B-Instruct_base"`). Defaults to `"model"`. - export\_format - Controls which export artifacts are written. `"v2"` (default) writes only the `sim.onnx.export()` artifacts. `"v1"` writes only the deprecated `quantsim.export()` artifacts. `"all"` writes both. - generator - Optional pre-built `LLMGenerator` for static-shape forward passes. Level-3 Python API users can pass a generator instance directly here instead of letting the recipe build one internally. - *field* attention\_mask\_min*: int* *= -100* - - *field* bypass\_adapted\_forward*: bool* *= False* - - *field* config\_overrides*: Dict[str, Any]* *[Optional]* - - *field* eaglet\_config*: Optional[Dict[str, Any]]* *= None* - - *field* export\_format*: Literal['v1', 'v2', 'all']* *= 'v2'* - - *field* filename\_prefix*: Optional[str]* *= None* - - *field* model\_preparation\_path*: Optional[str]* *= None* - - *field* recipe\_config*: Optional[Union[str, Dict[str, Any]]]* *= None* - - *field* recipe\_name*: str* *[Required]* - - *field* technique\_kwargs*: Dict[str, Any]* *[Optional]* - - model\_post\_init(*context: Any*, */*) → None - We need to both initialize private attributes and call the user-defined model\_post\_init method. - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.quantization\_stage.QuantizationInput - Bases: [`StageInput`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageInput) Upstream model and tokenizer provided by the model loading stage. - *field* config*: Optional[Any]* *= None* - - *field* model*: torch.nn.Module* *[Required]* - - *field* tokenizer*: Any | None* *= None* - - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.quantization\_stage.QuantizationOutput - Bases: [`StageOutput`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageOutput) Output from the quantization stage. - model - The reauthored model wrapping `quantsim.model`. - quantsim - AIMET QuantizationSimModel; `None` for optimization-only recipes or when loading from cache. - quantizer - The `AIMETQuantizer` instance used during quantization. Retained so that `export()` can delegate to `AIMETQuantizer.export()`. `None` when loading from cache. - tokenizer - Passed through from upstream stage. - config - HuggingFace model config passed through from upstream stage. - generator - Optional LLMGenerator wrapping the quantized model for evaluation. Set when the model was prepared via `prepare_llm` so downstream evaluation receives a ready-to-use model without signature inspection. - export\_format - The export format used when writing artifacts — `"v2"`, `"v1"`, or `"all"`. Populated by `export()` and `load_from_cache()` so downstream stages know which layout was written. - model\_path - Path to exported ONNX model; populated after export or cache restore. - encodings\_path - Path to exported encodings file; populated after export or cache restore. - tokenizer\_path - Path to tokenizer directory; populated after export or cache restore. - config\_path - Path to exported config.json; populated after export or cache restore. - fp\_model - The untouched floating-point model passed into quantization. Forwarded so downstream stages (e.g. draft-model loading) can build from the original HF-style model, while `model` holds the quantization product itself. - *field* config*: Optional[Any]* *= None* - - *field* config\_path*: Optional[str]* *= None* - - *field* encodings\_path*: Optional[str]* *= None* - - *field* export\_format*: Literal['v1', 'v2', 'all']* *= 'v2'* - - *field* fp\_model*: Optional[torch.nn.Module]* *= None* - - *field* generator*: Optional[Any]* *= None* - - *field* hidden\_states\_cache\_path*: Optional[str]* *= None* - - *field* model*: Optional[torch.nn.Module]* *= None* - - *field* model\_path*: Optional[str]* *= None* - - *field* quantizer*: Optional[AIMETQuantizer]* *= None* - - *field* quantsim*: Optional[Any]* *= None* - - *field* tokenizer*: Any | None* *= None* - - *field* tokenizer\_path*: Optional[str]* *= None* - - *class* qairt.experimental.pipeline.torch.llm.stages.quantization\_stage.QuantizationStage - Bases: [`Stage`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.Stage)[`QuantizationInput`, `QuantizationConfig`, `QuantizationOutput`] Runs a named AIMET quantization recipe and produces a quantized model artifact. - Config - alias of `QuantizationConfig` - Input - alias of `QuantizationInput` - Output - alias of `QuantizationOutput` - evaluate(*stage\_output: QuantizationOutput*, *config: QuantizationConfig*, *\*\*kwargs: Any*) → dict[str, float] - Run the configured evaluation metrics on the quantized model. - Parameters - - **stage\_output** – Output from the quantization stage (model, tokenizer, config, and optionally a pre-wrapped generator). - **config** – Stage configuration carrying `evaluator_config` and, via the pipeline context, the `cache_dir` default output location. - **\*\*kwargs** – Only `evaluator_config` is accepted, to override the one on the stage/pipeline config; it takes precedence when provided. Any other keyword argument raises `TypeError` (guards against silent typos). - Returns - The `{display_name: score}` mapping from `run_evaluation()`. - Raises - - **ValueError** – If no `evaluator_config` with metrics is configured, the tokenizer is missing, or sequence/context lengths are unavailable for wrapping the model. - **TypeError** – If an unexpected keyword argument is passed. - *classmethod* export(*stage\_output: QuantizationOutput*, *output\_path: Path*, *\*\*kwargs: Any*) → None - Export the quantized model to `output_path` (ONNX + encodings + tokenizer + config). Delegates to `QuantizationResult.export()` which handles ONNX export, tokenizer, and model config saving in one call. Populates `model_path`, `encodings_path`, `tokenizer_path`, and `config_path` on `stage_output` so downstream stages can consume them. - Parameters - - **stage\_output** – Output from this stage. - **output\_path** – Destination directory. - **\*\*kwargs** – `filename_prefix` (default `"model"`). `export_format`: override the export format; defaults to `stage_output.export_format` (`"v2"`, `"v1"`, or `"all"`). When `"all"`, both v2 ONNX artifacts and v1 encodings are written. - generate(*stage\_output: QuantizationOutput*, *config: QuantizationConfig*, *prompt: Union[str, list[dict[str, str]], Path]*, *\*\*kwargs: Any*) → Any - Run inference on the quantized model for quick accuracy checks post-quantization. - Parameters - - **stage\_output** – Output from the quantization stage. - **prompt** – Input text to generate from. Can be a string, a Path to a text file, or a list of chat messages (in which case the last message content is used). - **\*\*kwargs** – Must include `sequence_length` and `context_length`. Additional kwargs (e.g. `max_new_tokens`) forwarded to the generator. - *classmethod* get\_stage\_dependency() → [StageDependencies](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageDependencies) - Declare dependencies for this stage. Override in subclasses; default is no dependencies. - *classmethod* load\_from\_cache(*cache\_path: Path*, *\*\*kwargs: Any*) → QuantizationOutput - Restore a previously exported quantization output from `cache_path`. ## QuantizationOptStage Quantization Optimization Stage Applies a pre-quantization optimization recipe to a loaded model to improve quantization quality. The technique is selected by `recipe_name` and dispatched through the same `RecipeRegistry` used by `QuantizationStage`. - Supported recipes (registered in `defaults.py`): - - seq\_mse\_opt : SeqMSE weight optimization - spinquant : SpinQuant Hadamard rotation - ada\_scale\_hf : AdaScale HF-adapter optimization - gptaq\_hf : GPTAQ HF-adapter weight quantization - prefix\_quant : Prefix KV-cache pre-computation - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.quantization\_opt\_stage.QuantizationOptConfig - Bases: [`StageConfig`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageConfig) Configuration for the quantization optimization stage. - recipe\_name - Registry key for the optimization recipe to run (e.g. `"spinquant"`, `"ada_scale_hf"`). - context\_length - Max context length forwarded to the recipe. Falls back to `generator_config` if omitted. - sequence\_length - Sequence length forwarded to the recipe. Falls back to `generator_config` if omitted. - dataloader - Pre-built DataLoader for calibration. Not serializable — pass at runtime. - optimizer\_kwargs - Kwargs forwarded verbatim to the recipe’s `apply()`. - *field* context\_length*: Optional[int]* *= None* - - *field* dataloader*: Optional[Any]* *= None* - - *field* optimizer\_kwargs*: Dict[str, Any]* *[Optional]* - - *field* recipe\_name*: str* *[Required]* - - *field* sequence\_length*: Optional[int]* *= None* - - model\_post\_init(*context: Any*, */*) → None - We need to both initialize private attributes and call the user-defined model\_post\_init method. - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.quantization\_opt\_stage.QuantizationOptInput - Bases: [`StageInput`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageInput) Upstream model provided by the model loading stage. - *field* config*: Optional[Any]* *= None* - - *field* model*: torch.nn.Module* *[Required]* - - *field* tokenizer*: Any | None* *= None* - - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.quantization\_opt\_stage.QuantizationOptOutput - Bases: [`StageOutput`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageOutput) Output from the quantization optimization stage. - *field* config*: Optional[Any]* *= None* - - *field* model*: torch.nn.Module* *[Required]* - - *field* prepared\_model*: Optional[torch.nn.Module]* *= None* - - *field* quantsim*: Optional[Any]* *= None* - - *field* tokenizer*: Any | None* *= None* - - *class* qairt.experimental.pipeline.torch.llm.stages.quantization\_opt\_stage.QuantizationOptStage - Bases: [`Stage`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.Stage)[`QuantizationOptInput`, `QuantizationOptConfig`, `QuantizationOptOutput`] Applies a pre-quantization optimization recipe to a loaded model. - Config - alias of `QuantizationOptConfig` - Input - alias of `QuantizationOptInput` - Output - alias of `QuantizationOptOutput` - *classmethod* export(*stage\_output: QuantizationOptOutput*, *output\_path: Path*, *\*\*kwargs: Any*) → None - Export the optimized model. HuggingFace models (e.g. from ada\_scale\_hf, gptaq\_hf, prefix\_quant) are saved in HF format via `save_pretrained`. Plain torch modules (e.g. SpinQuant `prepared_model`, SeqMSE `quantsim.model`) are saved as a state-dict via `torch.save` because they do not carry HF config metadata. - *classmethod* get\_stage\_dependency() → [StageDependencies](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageDependencies) - Declare dependencies for this stage. Override in subclasses; default is no dependencies. - *classmethod* load\_from\_cache(*cache\_path: Path*, *\*\*kwargs: Any*) → QuantizationOptOutput - Restore a previously exported optimized model from `cache_path`. ## GenAIBuilderStage GenAI Builder Stage Builds an `LLMContainer` from ONNX/model artifacts using `GenAIBuilderFactory`. The stage can also run generation by creating a `T2TExecutor` from the container. - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.gen\_ai\_builder.builder\_stage.GenAIBuilderConfig - Bases: [`StageConfig`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageConfig) Stage config for container build: backend/targets and advanced builder options. - Validators - - `_coerce_compile_config` » `compile_config` - `_coerce_transform_config` » `transform_config` - `_validate_compile_args` » `all fields` - `_validate_max_workers` » `max_workers` - `_validate_transform_args` » `all fields` - *field* attach\_model\_for\_arn*: Optional[List[Tuple[int, str, Optional[str]]]]* *= None* - Attach secondary AR models: list of (arn, model\_path, encodings\_path) tuples. - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* calibration\_options*: Optional[dict[str, Any]]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* chat\_template*: Optional[str]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* compile\_backend\_extensions\_path*: Optional[FilePath]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* compile\_config*: Optional[[CompileConfig](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-api-compiler.html#qairt.api.compiler.config.CompileConfig)]* *= None* - - Validated by - - `_coerce_compile_config` - `_validate_compile_args` - `_validate_transform_args` - *field* compile\_options*: Optional[dict[str, Any]]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* conversion\_options*: Optional[dict[str, Any]]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* max\_workers*: Optional[int]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_max_workers` - `_validate_transform_args` - *field* multi\_graph*: Optional[bool]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* native\_kv*: Optional[bool]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* parallel\_build*: Optional[bool]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* prepare\_embedding\_lut*: Optional[bool]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* speculative\_config*: Optional[LadeBuilderConfig | EagletBuilderConfig | SsdBuilderConfig]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* transform\_config*: Optional[ModelTransformerConfig]* *= None* - - Validated by - - `_coerce_transform_config` - `_validate_compile_args` - `_validate_transform_args` - *field* transform\_options*: Optional[dict[str, Any]]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - *field* weight\_sharing*: Optional[bool]* *= None* - - Validated by - - `_validate_compile_args` - `_validate_transform_args` - model\_post\_init(*context: Any*, */*) → None - We need to both initialize private attributes and call the user-defined model\_post\_init method. - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.gen\_ai\_builder.builder\_stage.GenAIBuilderInput - Bases: [`StageInput`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageInput) Upstream model artifact paths provided by the immediate previous stage. - *field* config\_path*: Optional[str]* *= None* - - *field* model\_path*: str* *[Required]* - - *field* preconfigured\_builder*: Optional[[GenAIBuilderHTP](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-gen-ai-api-builders-htp.html#qairt.gen_ai_api.builders.gen_ai_builder_htp.GenAIBuilderHTP)]* *= None* - - *field* tokenizer\_path*: Optional[str]* *= None* - - *pydantic model* qairt.experimental.pipeline.torch.llm.stages.gen\_ai\_builder.builder\_stage.GenAIBuilderOutput - Bases: [`StageOutput`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageOutput) Build output containing the LLM container. - *field* container*: [LLMContainer](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-gen-ai-api-containers.html#qairt.gen_ai_api.containers.llm_container.LLMContainer)* *[Required]* - - *class* qairt.experimental.pipeline.torch.llm.stages.gen\_ai\_builder.builder\_stage.GenAIBuilderStage - Bases: [`Stage`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.Stage)[`GenAIBuilderInput`, `GenAIBuilderConfig`, `GenAIBuilderOutput`] Builds an LLMContainer via GenAIBuilderFactory and can run generation through its executor. - Config - alias of `GenAIBuilderConfig` - Input - alias of `GenAIBuilderInput` - Output - alias of `GenAIBuilderOutput` - artifact\_dirname(*config: GenAIBuilderConfig*, *key: str*) → str - Return a human-readable directory name: `_container_`. - *classmethod* export(*stage\_output: GenAIBuilderOutput*, *output\_path: Path*, *\*\*kwargs: Any*) → None - Export the built container artifacts to `output_path`. Uses `container.save()` so the layout is compatible with `LLMContainer.load()` in `load_from_cache()`. Device deployment (`container.export()` with a `target_device`) is a separate step outside the pipeline export path. - generate(*stage\_output: GenAIBuilderOutput*, *config: GenAIBuilderConfig*, *prompt: Union[str, List[Dict[str, str]], Path]*, *\*\*kwargs: Any*) → TextGenerationResult - Create a GenAI executor from the container and run text generation. - Supported `kwargs`: - device (Device | None): Target device for execution. clean\_up (bool): Clean temporary runtime artifacts after execution. Default `True`. prepare\_environment (bool): Prepare runtime environment before execution. Default `True`. qairt\_sdk\_root (str | None): Optional QAIRT SDK root override. lora\_config (Any): Optional LoRA runtime config forwarded to executor.generate(). - *classmethod* get\_stage\_dependency() → [StageDependencies](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-base.html#qairt.experimental.pipeline.torch.common.bases.stage.StageDependencies) - Declare dependencies for this stage. Override in subclasses; default is no dependencies. - *classmethod* load\_from\_cache(*cache\_path: Path*, *\*\*kwargs: Any*) → GenAIBuilderOutput - Restore a previously exported container from `cache_path`. Last Published: Jul 08, 2026 [Previous Topic PipelineFeatures.speculative\_decoding](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/qairt-pipeline-llm.md) [Next Topic Quantization Recipes](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/qairt-pipeline-quantization.md)