# Quantization Recipes

The quantization recipe system provides a registry of named recipes that
encapsulate complete quantization workflows (quantizer + optimizer chains).

## RecipeRegistry

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.registry.RecipeRegistry

    - Bases: `object`

Central registry for quantization recipes.

- *classmethod* get(*name: str*) → Type[[AIMETQuantizationRecipe](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)]

    - Get recipe class by name.

- Parameters

    - **name** – Recipe identifier

- Returns

    - Recipe class

- Raises

    - **ValueError** – If recipe not found

- *classmethod* get\_default\_config(*name: str*) → Optional[Path]

    - Get default config path for recipe.

- Parameters

    - **name** – Recipe identifier

- Returns

    - Path to default config file, or None if not registered

- *classmethod* list\_recipes() → list[dict]

    - List all registered recipes with metadata.

- Returns

    - List of dicts with ‘name’, ‘result\_type’, and ‘description’ keys.

- *classmethod* register(*name: str*, *recipe\_class: Type[[AIMETQuantizationRecipe](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)]*, *default\_config: Optional[Union[str, Path]] = None*)

    - Register a recipe with optional default config.

- Parameters

    - - **name** – Recipe identifier (e.g., “lpbq\_mse”)
- **recipe\_class** – Recipe class to register
- **default\_config** – Path to default YAML config file

## AIMETQuantizationRecipe

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe

    - Bases: `ABC`

Base class for AIMET Quantization Recipes.

A recipe combines optimization techniques (e.g., SpinQuant, AdaScale) with
quantization techniques (e.g., LPBQ, PCQ) to create complete quantization workflows.

Recipes use a nested parameter convention with maximum 2 levels:

<technique>: {<parameter>: <value>, ...}
    Copy to clipboard

Example:

seqmse:
        num_batches: 20
        num_candidates: 20
    lpbq:
        module_precisions:
          lm_head: "int8"
        tie_quantizers: [concat]
    Copy to clipboard

- *abstract* apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer: Optional[PreTrainedTokenizerBase] = None*, *dataloader: Optional[DataLoader] = None*, *\*\*recipe\_kwargs*) → qairt.experimental.pipeline.torch.llm.quantization.techniques.bases.definitions.QuantizationResult | qairt.experimental.pipeline.torch.llm.quantization.techniques.bases.definitions.OptimizationResult

    - Apply the quantization recipe.

- Parameters

    - - **model** – HuggingFace model to quantize.
- **context\_length** – Max context length for inputs.
- **sequence\_length** – Sequence length for dummy inputs.
- **tokenizer** – Optional tokenizer associated with the model.
- **dataloader** – Optional dataloader for calibration/optimization.
- **\*\*recipe\_kwargs** – Additional keyword arguments to forward to optimizers/quantizers.
Recipe kwargs use nested structure (max 2 levels):
`<technique>: {<parameter>: <value>, ...}`
Example: `seqmse={'num_batches': 20, 'num_candidates': 20}`

- Returns

    - QuantizationResult if recipe includes quantization step, or
OptimizationResult if recipe only performs optimization.

- *classmethod* apply\_from\_config(*config: Optional[Union[str, Dict]] = None*, *config\_overrides: Optional[Dict] = None*, *model: Optional[Module] = None*, *tokenizer: Optional[PreTrainedTokenizerBase] = None*, *dataloader: Optional[DataLoader] = None*) → qairt.experimental.pipeline.torch.llm.quantization.techniques.bases.definitions.QuantizationResult | qairt.experimental.pipeline.torch.llm.quantization.techniques.bases.definitions.OptimizationResult | None

    - Apply the quantization recipe entirely using configuration from a YAML file.

The configuration file must include:

- ‘model\_id’: Model identifier. Used to load model/tokenizer if not provided, and to
filter whether recipe should be applied if model is already provided.

The configuration file can optionally include:

- ‘dataset’: Automatically create a dataloader. Currently supports ‘wikitext’.
If specified and dataloader is None, a dataloader will be created automatically.

- Parameters

    - - **config** – Path to recipe configuration YAML file or config dict. If None, uses default config from class.
- **config\_overrides** – Optional dictionary to override specific configuration parameters.
- **model** – Optional HuggingFace model to quantize. If None, will be loaded using
‘model\_id’ from config.
- **tokenizer** – Optional tokenizer. If None, will be loaded using ‘model\_id’ from config.
- **dataloader** – Optional dataloader for calibration/optimization.

- Returns

    - QuantizationResult or OptimizationResult with recipe applied, or None if model\_id filter does not match.

- Raises

    - **ValueError** – If model\_id is not in config and model is None.

- build\_use\_case\_quantsim(*inner\_model: Module*) → QuantizationSimModel

    - Build a QuantSim over `inner_model` configured like the base pass.

Stateful — must be called after [`apply()`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe.apply), which populates
`_quantizer` and the `_base_*` QuantSim configuration.  Rebuilds a
QuantSim over the (LoRA-attached) `inner_model` using the same quantizer
and QuantSim configuration as the base pass, then applies the
technique-specific topology hook (LPBQ blockwise / no-op).

- Parameters

    - **inner\_model** – The (LoRA-attached) prepared model to quantize.

- Returns

    - The freshly-created QuantSim with its technique topology applied. It
is NOT yet calibrated — the caller must call [`calibrate_use_case()`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe.calibrate_use_case)
(after any LoRA-specific quantizer setup) before [`export_use_case()`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe.export_use_case).

- calibrate\_use\_case(*sim: QuantizationSimModel*, *calibration\_dataloader: DataLoader*, *num\_calibration\_batches: int*) → None

    - Calibrate `sim` with generator-driven forward passes, then clip weights.

Must run AFTER any LoRA-specific quantizer setup / freeze policy so the
computed encodings reflect the final quantizer topology.

- config*: Optional[Union[Dict[str, Any], str]]*  *= None*

    - 

- export\_use\_case(*sim: QuantizationSimModel*, *export\_dir: Path*, *filename\_prefix: str*) → tuple[Path, Path]

    - Export the calibrated `sim` to ONNX + encodings.

Must run LAST — after topology, LoRA quantizer setup, the freeze/reoptimize
branch, and calibration — so the exported artifacts carry the final
encodings.

- Returns

    - `(onnx_path, encodings_path)` for the exported artifacts.

- reoptimize\_use\_case(*sim: QuantizationSimModel*, *dataloader: DataLoader*) → None

    - Re-run the technique optimizer (e.g. SeqMSE) over `sim`.

Public seam delegating to the `_reoptimize_use_case()` hook so callers
can invoke re-optimization at the right point in the sequence.

- result\_type

    - alias of `QuantizationResult`

- supports\_post\_prepare\_eval*: bool*  *= False*

    -

## Module-Level Functions

- qairt.experimental.pipeline.torch.llm.quantization.recipes.registry.list\_recipes() → list[dict]

    - List all available recipes with metadata.

- Returns

    - List of dicts with ‘name’, ‘result\_type’, and ‘description’ keys.

- qairt.experimental.pipeline.torch.llm.quantization.recipes.registry.load\_recipe(*name: str*, *\**, *config\_overrides: Optional[Dict[str, Any]] = None*)

- qairt.experimental.pipeline.torch.llm.quantization.recipes.registry.load\_recipe(*name: str*, *config: Optional[Union[str, Path, Dict[str, Any]]]*, *config\_overrides: Optional[Dict[str, Any]] = None*)

    - Load a recipe by name, returning a callable that delegates to apply\_from\_config.

- Parameters

    - - **name** – Recipe identifier (e.g., “lpbq\_seqmse”)
- **config** – Optional config source:
- `None`: use the recipe class default config.
- `dict`: dict of overrides applied on top of the class default config.
- `str`/`Path`: load YAML and resolve `name` as a top-level key.
- **config\_overrides** – Optional dict to override default config values

- Returns

    - A functools.partial wrapping recipe\_class.apply\_from\_config

- Raises

    - **ValueError** – If recipe not found

Example

recipe = load\_recipe(“lpbq\_seqmse”)
result = recipe(model=model, tokenizer=tok, dataloader=loader)

- qairt.experimental.pipeline.torch.llm.quantization.recipes.registry.register\_recipe(*name: str*, *default\_config: Optional[Union[str, Path]] = None*)

    - Decorator to register a recipe.

Example:

@register_recipe("lpbq_mse", default_config="recipes/configs/lpbq_mse.yaml")
    class LPBQ_SeqMSE_Recipe(AIMETQuantizationRecipe):
        ...
    Copy to clipboard

- Parameters

    - - **name** – Recipe identifier
- **default\_config** – Optional path to default YAML config file. When omitted,
it is inferred as `<recipe_file_parent>/configs/<name>.yaml`.

- Returns

    - Decorator function

## Default Recipes

Default concrete recipe implementations.

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.AdaScaleHfRecipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

AdaScale HF-adapter optimization (adaptive weight scaling).

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer=None*, *dataloader=None*, *\*\*recipe\_kwargs*) → OptimizationResult

    - Apply the quantization recipe.

- Parameters

    - - **model** – HuggingFace model to quantize.
- **context\_length** – Max context length for inputs.
- **sequence\_length** – Sequence length for dummy inputs.
- **tokenizer** – Optional tokenizer associated with the model.
- **dataloader** – Optional dataloader for calibration/optimization.
- **\*\*recipe\_kwargs** – Additional keyword arguments to forward to optimizers/quantizers.
Recipe kwargs use nested structure (max 2 levels):
`<technique>: {<parameter>: <value>, ...}`
Example: `seqmse={'num_batches': 20, 'num_candidates': 20}`

- Returns

    - QuantizationResult if recipe includes quantization step, or
OptimizationResult if recipe only performs optimization.

- config*: Dict[str, Any] | str | None*  *= 'configs/ada\_scale\_hf.yaml'*

    - 

- result\_type

    - alias of `OptimizationResult`

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.AdaScale\_GPTAQ\_Recipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

Recipe combining AdaScale optimization with GPTAQ optimization.

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer: Optional[PreTrainedTokenizerBase] = None*, *dataloader: Optional[DataLoader] = None*, *generator: Optional[Any] = None*, *\*\*recipe\_kwargs*) → OptimizationResult

    - Apply the quantization recipe.

- Parameters

    - - **model** – HuggingFace model to quantize.
- **context\_length** – Max context length for inputs.
- **sequence\_length** – Sequence length for dummy inputs.
- **tokenizer** – Optional tokenizer associated with the model.
- **dataloader** – Optional dataloader for calibration/optimization.
- **\*\*recipe\_kwargs** – Additional keyword arguments to forward to optimizers/quantizers.
Recipe kwargs use nested structure (max 2 levels):
`<technique>: {<parameter>: <value>, ...}`
Example: `seqmse={'num_batches': 20, 'num_candidates': 20}`

- Returns

    - QuantizationResult if recipe includes quantization step, or
OptimizationResult if recipe only performs optimization.

- config*: Dict[str, Any] | str | None*  *= 'quantization/adascale\_gptaq.yaml'*

    - 

- result\_type

    - alias of `OptimizationResult`

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.CalibratorRecipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

Plain Calibrator recipe for calibration-and-export-oriented PTQ flows.

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer: Optional[PreTrainedTokenizerBase] = None*, *dataloader: Optional[DataLoader] = None*, *generator: Optional[Any] = None*, *\*\*recipe\_kwargs*) → QuantizationResult

    - Apply the existing Calibrator technique through the recipe registry.

This recipe intentionally performs only calibration/encoding generation.
It does not run LPBQ, SeqMSE, AdaScale, GPTAQ, or any other optimization
pass.  Keeping it as a normal recipe preserves the YAML pipeline contract:
QuantizationStage selects by recipe\_name and remains unaware of raw
technique classes.

- config*: Dict[str, Any] | str | None*  *= 'quantization/calibrator.yaml'*

    - 

- result\_type

    - alias of `QuantizationResult`

- supports\_post\_prepare\_eval*: bool*  *= True*

    -

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.GPTAQHfRecipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

GPTAQ weight quantization pass.

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer=None*, *dataloader=None*, *\*\*recipe\_kwargs*) → OptimizationResult

    - Apply the quantization recipe.

- Parameters

    - - **model** – HuggingFace model to quantize.
- **context\_length** – Max context length for inputs.
- **sequence\_length** – Sequence length for dummy inputs.
- **tokenizer** – Optional tokenizer associated with the model.
- **dataloader** – Optional dataloader for calibration/optimization.
- **\*\*recipe\_kwargs** – Additional keyword arguments to forward to optimizers/quantizers.
Recipe kwargs use nested structure (max 2 levels):
`<technique>: {<parameter>: <value>, ...}`
Example: `seqmse={'num_batches': 20, 'num_candidates': 20}`

- Returns

    - QuantizationResult if recipe includes quantization step, or
OptimizationResult if recipe only performs optimization.

- config*: Dict[str, Any] | str | None*  *= 'configs/gptaq\_hf.yaml'*

    - 

- result\_type

    - alias of `OptimizationResult`

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.LPBQ\_SeqMSE\_Recipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

Recipe combining LPBQ quantization with SeqMSE optimization.

- Execution order:
    - 1. Create quantsim with module precisions
2. Apply LPBQ blockwise topology (before any optimization)
3. Run SeqMSE (optimizes with LPBQ topology in place)
4. Calibrate / compute encodings (directly on quantsim)

- Uses nested parameter structure (max 2 levels):
    - seqmse: {num\_batches: 20, num\_candidates: 20}
lpbq: {module\_precisions: {kv\_cache: {weight\_precision: int8}}, tie\_quantizers: [concat]}

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer: Optional[PreTrainedTokenizerBase] = None*, *dataloader: Optional[DataLoader] = None*, *seqmse\_dataloader: Optional[DataLoader] = None*, *generator: Optional[Any] = None*, *skip\_base\_optimization: bool = False*, *\*\*recipe\_kwargs*) → QuantizationResult

    - Apply the quantization recipe.

- Parameters

    - - **model** – HuggingFace model to quantize.
- **context\_length** – Max context length for inputs.
- **sequence\_length** – Sequence length for dummy inputs.
- **tokenizer** – Optional tokenizer associated with the model.
- **dataloader** – Optional dataloader for calibration/optimization.
- **\*\*recipe\_kwargs** – Additional keyword arguments to forward to optimizers/quantizers.
Recipe kwargs use nested structure (max 2 levels):
`<technique>: {<parameter>: <value>, ...}`
Example: `seqmse={'num_batches': 20, 'num_candidates': 20}`

- Returns

    - QuantizationResult if recipe includes quantization step, or
OptimizationResult if recipe only performs optimization.

- config*: Dict[str, Any] | str | None*  *= 'quantization/lpbq\_seqmse.yaml'*

    - 

- supports\_post\_prepare\_eval*: bool*  *= True*

    -

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.PrefixQuantRecipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

Prefix KV-cache pre-computation optimization.

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer=None*, *dataloader=None*, *\*\*recipe\_kwargs*) → OptimizationResult

    - Apply the quantization recipe.

- Parameters

    - - **model** – HuggingFace model to quantize.
- **context\_length** – Max context length for inputs.
- **sequence\_length** – Sequence length for dummy inputs.
- **tokenizer** – Optional tokenizer associated with the model.
- **dataloader** – Optional dataloader for calibration/optimization.
- **\*\*recipe\_kwargs** – Additional keyword arguments to forward to optimizers/quantizers.
Recipe kwargs use nested structure (max 2 levels):
`<technique>: {<parameter>: <value>, ...}`
Example: `seqmse={'num_batches': 20, 'num_candidates': 20}`

- Returns

    - QuantizationResult if recipe includes quantization step, or
OptimizationResult if recipe only performs optimization.

- config*: Dict[str, Any] | str | None*  *= 'configs/prefix\_quant.yaml'*

    - 

- result\_type

    - alias of `OptimizationResult`

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.SeqMSEOptimizationRecipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

SeqMSE weight optimization

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer=None*, *dataloader=None*, *\*\*recipe\_kwargs*) → OptimizationResult

    - Apply the quantization recipe.

- Parameters

    - - **model** – HuggingFace model to quantize.
- **context\_length** – Max context length for inputs.
- **sequence\_length** – Sequence length for dummy inputs.
- **tokenizer** – Optional tokenizer associated with the model.
- **dataloader** – Optional dataloader for calibration/optimization.
- **\*\*recipe\_kwargs** – Additional keyword arguments to forward to optimizers/quantizers.
Recipe kwargs use nested structure (max 2 levels):
`<technique>: {<parameter>: <value>, ...}`
Example: `seqmse={'num_batches': 20, 'num_candidates': 20}`

- Returns

    - QuantizationResult if recipe includes quantization step, or
OptimizationResult if recipe only performs optimization.

- config*: Dict[str, Any] | str | None*  *= 'configs/seq\_mse\_opt.yaml'*

    - 

- result\_type

    - alias of `OptimizationResult`

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.SpinQuant\_AdaScale\_Recipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

Recipe combining SpinQuant and AdaScale optimizations.

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer: Optional[PreTrainedTokenizerBase] = None*, *dataloader: Optional[DataLoader] = None*, *generator: Optional[Any] = None*, *\*\*recipe\_kwargs*) → OptimizationResult

    - Apply SpinQuant followed by AdaScale optimization.

- config*: Dict[str, Any] | str | None*  *= 'quantization/spinquant\_adascale.yaml'*

    - 

- result\_type

    - alias of `OptimizationResult`

- *class* qairt.experimental.pipeline.torch.llm.quantization.recipes.defaults.SpinQuant\_Recipe

    - Bases: [`AIMETQuantizationRecipe`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.quantization.recipes.recipe.AIMETQuantizationRecipe)

Recipe applying SpinQuant optimization only (no quantization).

- Uses nested parameter structure (max 2 levels):
    - spinquant: {} (currently no parameters)

- apply(*model: Module*, *context\_length: int = 4096*, *sequence\_length: int = 2048*, *tokenizer: Optional[PreTrainedTokenizerBase] = None*, *dataloader: Optional[DataLoader] = None*, *\*\*recipe\_kwargs*) → OptimizationResult

    - Apply the quantization recipe.

- Parameters

    - - **model** – HuggingFace model to quantize.
- **context\_length** – Max context length for inputs.
- **sequence\_length** – Sequence length for dummy inputs.
- **tokenizer** – Optional tokenizer associated with the model.
- **dataloader** – Optional dataloader for calibration/optimization.
- **\*\*recipe\_kwargs** – Additional keyword arguments to forward to optimizers/quantizers.
Recipe kwargs use nested structure (max 2 levels):
`<technique>: {<parameter>: <value>, ...}`
Example: `seqmse={'num_batches': 20, 'num_candidates': 20}`

- Returns

    - QuantizationResult if recipe includes quantization step, or
OptimizationResult if recipe only performs optimization.

- config*: Dict[str, Any] | str | None*  *= 'quantization/spinquant.yaml'*

    - 

- result\_type

    - alias of `OptimizationResult`

## Test Vector Generation

- *class* qairt.experimental.pipeline.torch.common.configs.TestVectorConfig(*\**, *num\_test\_vectors: int = 1*, *slice\_num: int = 1*, *split\_candidate\_layers: list[str] = None*, *output\_index\_mapping: dict[int, str] = None*, *dataset: Optional[Union[str, dict[str, Any]]] = None*, *output\_dir: str*, *sequence\_length: Optional[int] = None*, *context\_length: Optional[int] = None*, *tokenizer: Optional[Any] = None*)

    - Bases: `BaseModel`

Configuration for golden test vector generation after quantization.

Test vectors pair one sample input with the model’s floating-point output
(`fp_<index>.pkl`) and quantized output (`qt_<index>.pkl`), plus
input/output tensors recorded at regex-selected internal layers. They are
replayed on-device to localize where the on-device path diverges.

- *field* context\_length*: Optional[int]*  *= None*

    - Overrides the generator’s total context window size.

- *field* dataset*: Optional[str | dict[str, Any]]*  *= None*

    - Dataset to draw samples from when no dataloader is passed; same string-or-dict shape the recipes accept. `None` defaults to wikitext.

- model\_computed\_fields*: ClassVar[dict[str, ComputedFieldInfo]]*  *= {}*

    - A dictionary of computed field names and their corresponding <cite>ComputedFieldInfo</cite> objects.

- model\_config*: ClassVar[ConfigDict]*  *= {'arbitrary\_types\_allowed': True}*

    - Configuration for the model, should be a dictionary conforming to [<cite>ConfigDict</cite>][pydantic.config.ConfigDict].

- model\_fields*: ClassVar[dict[str, FieldInfo]]*  *= {'context\_length': FieldInfo(annotation=Union[int, NoneType], required=False, default=None, description="Overrides the generator's total context window size."), 'dataset': FieldInfo(annotation=Union[str, dict[str, Any], NoneType], required=False, default=None, description='Dataset to draw samples from when no dataloader is passed; same string-or-dict shape the recipes accept. ``None`` defaults to wikitext.'), 'num\_test\_vectors': FieldInfo(annotation=int, required=False, default=1, description='Number of dataloader samples to record.'), 'output\_dir': FieldInfo(annotation=str, required=True, description='Destination for the written vectors. ``fp\_N.pkl`` / ``qt\_N.pkl`` are written under ``&lt;output\_dir&gt;/test\_vectors/``.'), 'output\_index\_mapping': FieldInfo(annotation=dict[int, str], required=False, default\_factory=&lt;lambda&gt;, description="Maps each model output position to a name used as the key in the saved pickle. Its length must equal the model's actual output count."), 'sequence\_length': FieldInfo(annotation=Union[int, NoneType], required=False, default=None, description="Overrides the generator's autoregressive window size. Needed on the technique-level path where no export generator exists."), 'slice\_num': FieldInfo(annotation=int, required=False, default=1, description='Index of the autoregressive window to record. Windows before it are run first to warm up the KV cache. ``0`` records the first window with no warm-up (not covered by legacy parity).'), 'split\_candidate\_layers': FieldInfo(annotation=list[str], required=False, default\_factory=&lt;lambda&gt;, description="Regex patterns matched against module \*names\*; every matching layer's inputs and outputs are recorded."), 'tokenizer': FieldInfo(annotation=Union[Any, NoneType], required=False, default=None, description='Explicit tokenizer for the technique-level path, where the result carries none.', exclude=True)}*

    - Metadata about the fields defined on the model,
mapping of field names to [<cite>FieldInfo</cite>][pydantic.fields.FieldInfo].

This replaces <cite>Model.__fields__</cite> from Pydantic V1.

- *field* num\_test\_vectors*: int*  *= 1*

    - Number of dataloader samples to record.

- *field* output\_dir*: str*  *[Required]*

    - Destination for the written vectors. `fp_N.pkl` / `qt_N.pkl` are written under `<output_dir>/test_vectors/`.

- *field* output\_index\_mapping*: dict[int, str]*  *[Optional]*

    - Maps each model output position to a name used as the key in the saved pickle. Its length must equal the model’s actual output count.

- *field* sequence\_length*: Optional[int]*  *= None*

    - Overrides the generator’s autoregressive window size. Needed on the technique-level path where no export generator exists.

- *field* slice\_num*: int*  *= 1*

    - Index of the autoregressive window to record. Windows before it are run first to warm up the KV cache. `0` records the first window with no warm-up (not covered by legacy parity).

- *field* split\_candidate\_layers*: list[str]*  *[Optional]*

    - Regex patterns matched against module *names*; every matching layer’s inputs and outputs are recorded.

- *field* tokenizer*: Optional[Any]*  *= None*

    - Explicit tokenizer for the technique-level path, where the result carries none.

Golden test vector generation for on-device validation.

- *class* qairt.experimental.pipeline.torch.llm.test\_vectors.QuantsimFromModel(*model: Module*)

    - Bases: `object`

Minimal `QuantizationSimModel` surface over a cache-restored quantized model.

On a cache hit the pipeline restores the quantized module tree but not the
sim object. The recorder only touches `sim.model` and `sim.qmodules()`,
so supplying those two is sufficient — encodings are already present as
registered `nn.Parameter` instances on the restored quantizer modules.

`qmodules()` deliberately yields the broad `BaseQuantizationMixin` set;
`reset_kernels()` narrows it to `QuantizationMixin` itself.

- qmodules() → Generator[Module, None, None]

    -

- qairt.experimental.pipeline.torch.llm.test\_vectors.generate\_test\_vectors(*sim: Any*, *model\_inputs: Any*, *output\_dir: str*, *batch\_index: int*, *test\_vector\_layers: Iterable[str]*, *idx\_to\_name\_output\_dict: Optional[Dict[int, str]] = None*) → str

    - Record `fp` and `qt` test vectors for one prepared input.

Runs two passes over `sim`: `fp` with all quantizers removed and custom
kernels reset, then `qt` with quantizers active. Both record the inputs
and outputs of every layer whose name matches `test_vector_layers`, and
write `{fp,qt}_<batch_index>.pkl` under `<output_dir>/test_vectors/`.

- Parameters

    - - **sim** – A `QuantizationSimModel` or a [`QuantsimFromModel`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.llm.test_vectors.QuantsimFromModel).
- **model\_inputs** – Prepared, static-shape model inputs.
- **output\_dir** – Parent directory; a `test_vectors` subdirectory is created.
- **batch\_index** – Index distinguishing multiple test vectors.
- **test\_vector\_layers** – Regexes matched against module names.
- **idx\_to\_name\_output\_dict** – Maps each output position to a name. Defaults
to `{0: "logits", 1: "past_key_values"}`.

- Returns

    - The directory the vectors were written to.

- Raises

    - **ValueError** – If the model’s output count differs from the mapping’s length.

- qairt.experimental.pipeline.torch.llm.test\_vectors.run\_test\_vector\_generation(*sim: Any*, *generator: Any*, *tokenizer: Any*, *dataloader: DataLoader*, *test\_vector\_config: [TestVectorConfig](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.common.configs.TestVectorConfig)*, *output\_dir: str*) → Optional[str]

    - Record test vectors for the first `num_test_vectors` dataloader samples.

For each sample the input is sliced into autoregressive windows. When
`slice_num > 0` the generator runs over the prefix spanning the first
`slice_num` windows to build up the KV cache, then the window at index
`slice_num` is prepared and recorded. `slice_num == 0` records the first
window with no warm-up.

- Parameters

    - - **sim** – A `QuantizationSimModel` or a
`QuantsimFromModel` over a restored model.
- **generator** – Generator wrapping the *quantized* model; supplies
`sequence_length`, `context_length` and the input preparation.
- **tokenizer** – Tokenizer, used for the pad token when preparing inputs.
- **dataloader** – Source of input samples.
- **test\_vector\_config** – Knobs; see
[`TestVectorConfig`](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html#qairt.experimental.pipeline.torch.common.configs.TestVectorConfig).
- **output\_dir** – Parent directory; vectors land in `<output_dir>/test_vectors/`.

- Returns

    - The directory the vectors were written to, or `None` if no vectors were
recorded (empty dataloader or every sample skipped due to `slice_num`
being out of range).

- Raises

    - **ValueError** – If `slice_num` is negative, or the model’s output count
    does not match `output_index_mapping`.

Last Published: Oct 23, 2026

[Previous Topic
QcAutoModelForImageTextToText](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/qairt-pipeline-stages.md)