# Test Vector Generation

The quantization stage can record golden **test vectors** — paired floating-point
and quantized reference outputs for a handful of calibration samples — so that an
on-device run can be replayed and compared against a known-good reference to
localize where it diverges.

A test vector pairs:

- `fp_<index>.pkl` — the floating-point model’s output and recorded internal
layer activations for sample `<index>`.
- `qt_<index>.pkl` — the quantized model’s output and recorded internal layer
activations for the same sample.

Both pickles are written to `<output_dir>/test_vectors/`.

The feature is opt-in and off by default. It’s only available on the LLM
`QuantizationStage`; the draft-model stage (`DMQuantizationStage`) doesn’t
support it.

## Enabling via Programmatic Config

Set `test_vector_config` on the quantization stage’s config:

from qairt.experimental.pipeline.torch.common.configs import TestVectorConfig
    
    test_vector_config = TestVectorConfig(
        output_dir="/path/to/output",
        num_test_vectors=2,
        slice_num=1,
    )
    Copy to clipboard

Pass it via the recipe’s per-stage overrides, e.g. in an `LLMPipelineConfig`:

stage_info={
        "quantization": {
            "test_vector_config": {
                "output_dir": "/path/to/output",
                "num_test_vectors": 2,
            },
        },
    }
    Copy to clipboard

When set, `QuantizationStage` generates test vectors automatically immediately
after quantization completes, provided the recipe produced a
`QuantizationResult` (recipes that produce a plain `OptimizationResult` are
skipped). Generation failures are logged as warnings rather than failing the
stage, because this is a side-output of quantization, not the primary result.

## Generating On Demand (Cache-Restored Stages)

Test vectors can also be generated after the fact from a cache-restored
`QuantizationOutput`, without re-running quantization:

stage.generate_test_vectors(stage_output, config, test_vector_config)
    Copy to clipboard

If `stage_output.quantsim` is unavailable (the common case on cache
restore), the stage wraps `stage_output.model` in `QuantsimFromModel` — a
minimal shim exposing `.model` and `.qmodules()` so the recorder can run
without a real `QuantizationSimModel`. This path additionally requires
`stage_output.generator` to be present; on cache restore, pass a
`generator_config` with both `sequence_length` and `context_length` set
so the generator can be rebuilt. A missing config, model, generator, or
tokenizer raises `ValueError`/`RuntimeError`, as does recording zero
vectors.

API Reference

See the [Quantization API reference](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-quantization.html)
for the full `TestVectorConfig` field reference and the `test_vectors`
module functions (`generate_test_vectors()`, `run_test_vector_generation()`,
`QuantsimFromModel`).

Last Published: Oct 05, 2026

[Previous Topic
Advanced Flow (Building Blocks)](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/lora.md) [Next Topic
Advanced Usage](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/pipeline_expert_usage.md)