# Eaglet

The pipeline supports **speculative decoding** via the Eaglet feature. A lightweight
Draft Model (DM) proposes candidate tokens in a tree structure; the Target Model (TM)
verifies and accepts or rejects them in one forward pass, increasing throughput without
changing output distribution.

**Validated models:** Qwen3-4B.

## Two-Track Topology

Enabling Eaglet adds two DM stages that run in parallel with the TM track:

TM track:  model_loader → quantization   ─┐
    DM track:  dm_model_loader → dm_quantization ─┤→ genai_builder
    Copy to clipboard

The `genai_builder` stage receives artifacts from both tracks and compiles the combined
TM + DM container for on-device execution.

## Approach 1: YAML Recipe

Pre-built recipes are provided for validated models (e.g.
`recipes/qwen3/Qwen--Qwen3-4B-eaglet-v2.yaml`). Enable Eaglet by setting
`features.speculative_decoding.type: eaglet` and adding a top-level `eaglet:` block:

features:
      speculative_decoding:
        type: eaglet
    
    eaglet:
      eaglet_version: v2
      draft_safetensors_path: /path/to/pretrained_dm_weights.safetensors
      trimmed_vocabulary_path: /path/to/trimmed_vocab.json
      num_draft_layers: 1
      num_calibration_batches: 20
      calibration_dataset: wikitext
      capture_hidden_states: true
      hidden_states_cache_dir: ./hidden_states_cache
      draft_top_k: 8
      draft_depth: 5
      total_draft_tokens: 31
    
    stages:
      model_loader:
        model_reauthoring: true
        apply_default_adaptations: true
        execution_environment: gpu
      quantization:
        recipe_name: lpbq_seqmse
      dm_model_loader: {}        # inherits config from the eaglet block above
      dm_quantization:
        recipe_name: eaglet_draft_v2
      genai_builder:
        native_kv: true
    Copy to clipboard

Then run:

from qairt.experimental.pipeline.torch.llm.pipeline import LLMPipeline
    
    pipe = LLMPipeline.from_pretrained(
        "Qwen/Qwen3-4B",
        recipe="Qwen--Qwen3-4B-eaglet-v2.yaml",
    )
    pipe.construct()
    Copy to clipboard

## Approach 2: Programmatic Config

Load from a recipe and override individual eaglet parameters before calling
`construct()`. The DM stages expose their eaglet config as a plain dict via
`stage.config.eaglet_config`:

from qairt.experimental.pipeline.torch.llm.pipeline import LLMPipeline
    
    pipe = LLMPipeline.from_pretrained(
        "Qwen/Qwen3-4B",
        recipe="Qwen--Qwen3-4B-eaglet-v2.yaml",
    )
    
    # Override DM construction config
    pipe.stages.dm_model_loader.config.eaglet_config["num_draft_layers"] = 2
    
    # Override DM calibration config
    pipe.stages.dm_quantization.config.eaglet_config["num_calibration_batches"] = 64
    pipe.stages.dm_quantization.config.eaglet_config["draft_top_k"] = 4
    
    pipe.construct()
    Copy to clipboard

## `EagletV2Config` Reference

For a full field reference see the
[LLM Pipeline API docs](https://docs.qualcomm.com/doc/80-87189-2/topic/qairt-pipeline-llm.html).
Key fields: `eaglet_version`, `draft_safetensors_path`,
`trimmed_vocabulary_path`, `num_calibration_batches`, `capture_hidden_states`,
`hidden_states_cache_dir`, `draft_top_k`, `draft_depth`, `total_draft_tokens`.

For Qwen3.5 hybrid (SSM/GDN) target models, set `eaglet_version: "v2_Qwen35"`.
This activates `EagletV2Qwen35Config`, which defaults `export_inputs_embeds=True`
and enables GDN state checkpoint/replay.

## Block Efficiency (BE) Evaluation

Once the DM has been quantized, you can measure how well it accelerates generation
by running a **Block Efficiency** evaluation. Block Efficiency is the average
number of tokens accepted per speculation step — the higher the BE, the fewer TM
verification passes are needed per generated token.

Enable it by setting `eval_tm_arn` / `eval_dm_arn` (the `[prefill, generation]`
ARN pairs used to reconstruct quantized TM/DM models for evaluation) on the top-level
`eaglet:` block, and adding a `BE` entry to `evaluator_config.metrics`:

eaglet:
      eval_tm_arn: [128, 32]   # [prefill, generation]
      eval_dm_arn: [128, 8]
    
    evaluator_config:
      context_length: 4096
      metrics:
        - name: BE
          dataset_name: xlam
          dataset_kwargs:
            path: null  # Set to path of evaluation dataset if needed
    
    stages:
      dm_model_loader: {}
      dm_quantization:
        recipe_name: eaglet_draft_v2
    Copy to clipboard

Note

`pipe.evaluate()` runs evaluation on the pipeline’s last stage, so a BE
evaluation requires a pipeline whose last stage is `dm_quantization` — build
and run a recipe that stops there (as above), separate from the full recipe
that continues on to `genai_builder` for on-device deployment.

Then call `pipe.evaluate()` after construction:

pipe = LLMPipeline.from_pretrained(
        "Qwen/Qwen3-4B",
        recipe="Qwen--Qwen3-4B-eaglet-v2-eval.yaml",
    )
    pipe.construct()
    results = pipe.evaluate()
    # {"BE_xlam": 3.2}
    Copy to clipboard

Results are returned as a dict and also written to `evaluation_results.json` /
`.txt` under the pipeline’s output directory.

Note

BE evaluation reconstructs four QuantSim models (TM/DM × prefill/generation) and
runs live speculative decoding over the sampled dataset — expect roughly two
hours on a single GPU with the default `num_samples=100`.

Only `BE` metrics are evaluated at the DM quantization stage; other metrics (e.g.
standalone perplexity) aren’t meaningful for a draft model and are skipped.

Last Published: Oct 05, 2026

[Previous Topic
Advanced Features](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/index_advanced_features.md) [Next Topic
LoRA](https://docs.qualcomm.com/bundle/publicresource/80-87189-2/topics/lora.md)