# ONNX Runtime backend

The onnxruntime\_onnx (v1.18.1) backend uses the QAic EP (execution provider) for deploying ONNX graphs on Cloud AI inference accelerators.

## ONNX Runtime model repository

For onnxruntime configuration, platform should be set to
`onnxruntime_onnx`. The `use_qaic` parameter must be set to `true`.

### Cloud AI parameters

Parameters are user-provided key-value pairs which Triton will pass to backend runtime
environment as variables and can be used in the backend processing
logic.

- config : path for configuration file containing compiler options.
- device\_id : id of Cloud AI device on which inference is targeted. (not
mandatory as the server auto picks the available device)
- use\_qaic : flag to indicate to use QAic execution provider.
- share\_session : flag to enable the use of a single runtime session
object across model instances.

Sample `config.pbtxt`

name: "resnet_onnx"
    platform: "onnxruntime_onnx"
    max_batch_size : 16
    default_model_filename : "aic100/model.onnx"
    input [
      {
        name: "data"
        data_type: TYPE_FP32
        dims: [3, 224, 224 ]
      }
    ]
    output [
      {
        name: "resnetv18_dense0_fwd"
        data_type: TYPE_FP32
        dims: [1000]
      }
    ]
    parameters [
      {
        key: "config"
        value: { string_value: "1/aic100/resnet.yaml" }
      },
      {
        key: "device_id"
        value: { string_value: "0" }
      },
      {
        key: "use_qaic"
        value: { string_value: "true" }
      },
      {
        key: "share_session"
        value: { string_value: "true" }
      }
    ]
    instance_group [
      {
        count: 2
        kind: KIND_MODEL
      }
    ]
    Copy to clipboard

## Start Triton server

Start the Triton server within the Triton container with the model repository path.

/opt/tritonserver/bin/tritonserver --model-repository=</path/to/repository>
    Copy to clipboard

Last Published: Aug 25, 2026

[Previous Topic
QAic backend](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/qaic-backend.md) [Next Topic
Python Backend for LLMs](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/python-llm-backend.md)