# Model inference and deployment You can deploy a pre-trained model for inference on a Cloud AI platform using the following model inference and deployment workflow for ONNX. Exporting the model in ONNX format is strongly recommended due to operator support. ## Model inference and deployment workflow for ONNX The following image shows the stages and primary roles in the model inference and deployment workflow for ONNX. The workflow spans multiple stages, from model compilation to production serving, and different roles may have different entry points into the workflow. Page-1 Sheet.54 Sheet.53 Sheet.52 Sheet.51 Sheet.50 Sheet.48 Sheet.2 Pretrained model Pretrainedmodel Sheet.5 ML/AI Engineer ML/AI Engineer Sheet.7 Export and prepare the model Export and prepare the model Sheet.8 Sheet.10 ML/AI Engineer ML/AI Engineer Sheet.11 Compile the model Compile the model Sheet.12 Sheet.14 ML/AI Engineer ML/AI Engineer Sheet.15 Execute the QPC Execute the QPC Sheet.16 Sheet.22 MLOps/DevOps Engineer MLOps/DevOps Engineer Sheet.23 Serve the model Serve the model Sheet.24 Sheet.27 MLOps/DevOps Engineer MLOps/DevOps Engineer Sheet.28 Set up containers Set up containers Sheet.29 Sheet.32 Infrastructure / Platform Engineer Infrastructure / Platform Engineer Sheet.33 Deploy the model Deploy the model Sheet.34 Dynamic connector Dynamic connector.58 ONNX ONNX Dynamic connector.86 QPC* QPC* Dynamic connector.87 Dynamic connector.88 Dynamic connector.89 Skip to Serve if using prebuilt vLLM Docker image Skip to Serve if using prebuilt vLLM Docker image Dynamic connector.90 Sheet.92 *QAic program container *QAic program container Stages and primary roles in the model inference and deployment workflow The following table provides a brief description of each stage in the workflow and the primary role responsible for each section. Stages in the model inference and deployment workflow | Stage | Description | Primary role | | --- | --- | --- | | [Export and prepare the model](https://docs.qualcomm.com/doc/80-99100-3/topic/index_Export-the-model.html#export-the-model) | Export and prepare a pre-trained model as explained in this section to achieve the best performance and accuracy. | ML/AI Engineer | | [Compile the model](https://docs.qualcomm.com/doc/80-99100-3/topic/index_model-compilation.html#reference-to-compile-the-model) | Compilation of the prepared ONNX file generates a QAiC program container (QPC) which can be loaded and executed on Cloud AI devices. Compile using the appropriate parameters for your use case, then tune the parameters to get the best performance from the device based on your throughput and latency requirements. | MI/AI Engineer | | [Execute the model](https://docs.qualcomm.com/doc/80-99100-3/topic/index_model-execution.html) | This section details the different ways (CLI, C++, Python, ONNX Runtime) to run inferences on Cloud AI platforms and how to profile inferences at the system and device level. | ML/AI Engineer | | [Set up containers](https://docs.qualcomm.com/doc/80-99100-3/topic/index_Docker.html#reference-to-docker) | This section details how to build and configure Docker container images for running inference workloads. Build your own or select from a set of prebuilt images for different inference workflows. | MLOps/DevOps Engineer | | [Deploy the model](https://docs.qualcomm.com/doc/80-99100-3/topic/index_Kubernetes.html) | Use Kubernetes to deploy the container to Cloud AI devices. | Infrastructure/Platform Engineer | | [Serve the model](https://docs.qualcomm.com/doc/80-99100-3/topic/index_Model-Serving.html) | Expose the compiled model as an inference endpoint using vLLM, Triton or Text Generation Inference (TGI) toolkit. | MLOps/DevOps Engineer | ## Select your path Depending on your role, you may not need to complete every stage — use the following diagram to identify your starting point.
Workflow by role
%%{init: { 'themeVariables': { 'edgeLabelBackground': '#fff' } } }%% flowchart TD %% Global Styling classDef default fill:#fff,stroke:#000,color:#000,font-family:Roboto; A([Start]) --> B{What is your role?} %% Subgraph to force horizontal alignment of roles subgraph Roles [ ] direction LR C(ML/AI Engineer) D(MLOps/DevOps Engineer) E(Infrastructure/Platform Engineer) end B --> C B --> D B --> E %% ML/AI Path C --> F{Do you have a\pre-exported model?} F -->|No| G[Export the model] F -->|Yes| H[Compile the model] G -->|ONNX| H H -->|QPC| I[Execute the model] %% MLOps Path D --> J{Starting point?} J -->|Containers| K[Set up containers] J -->|Serving only| N[Serve the model] K --> L[Deploy the model] L --> N %% Infrastructure Path E --> L %% Style the subgraph to be invisible style Roles fill:none,stroke:none

Model inference workflow tasks by role

## Next steps - If you are a ML/AI engineer, start with [export and prepare the model](https://docs.qualcomm.com/doc/80-99100-3/topic/index_Export-the-model.html#export-the-model). - If you are a MLOps/DevOps engineer, see [Set up containers with Docker](https://docs.qualcomm.com/doc/80-99100-3/topic/index_Docker.html#reference-to-docker) and [Serve the model](https://docs.qualcomm.com/doc/80-99100-3/topic/index_Model-Serving.html). - If you are a Infrastructure/Platform engineer, see [Deploy the model with Kubernetes](https://docs.qualcomm.com/doc/80-99100-3/topic/index_Kubernetes.html). Last Published: Aug 25, 2026 [Previous Topic Verification](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/verification.md) [Next Topic Export the model](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/index_Export-the-model.md)