# Run an ONNX model on NPU using ORT ## Overview: Qualcomm AI Engine Direct (QNN) EP The QNN execution provider for ONNX runtime (ORT) enables hardware accelerated inference on Qualcomm chipsets using the Qualcomm AI Engine Direct SDK. The ONNX runtime QNN execution provider is commonly used on IOT devices with Qualcomm Snapdragon SoCs. You can create quantized ONNX models using ORT QDQ quantization tools or Qualcomm workflows that apply hardware-aware optimizations for deployment. For more information, see [ONNX Runtime (ORT) with QNN](https://onnxruntime.ai/docs/execution-providers/QNN-ExecutionProvider.html#qnn-execution-provider) ## Model preparation options ### Use a QDQ model Running a model with the QNN EP requires a quantize-dequantize (QDQ) ONNX model. This documentation uses prebuilt QDQ models from AI Hub, which are validated for QNN compatibility. You can also generate your own QDQ model using open source tools, such as the ONNX runtime quantization tool. VBackground-1 Solid Page-1 Rectangle.10 Host (Windows x86) Host (Windows x86) Rectangle.15 ORT ORT Rectangle.16 Q-DQ ONNX Q-DQONNX Rectangle.17 AIMET / ORT / GPTQ AIMET / ORT / GPTQ Rectangle.18 ORT session ORT session Rectangle.19 libQnnHtp.so libQnnHtp.so Rectangle.20 QNN EP QNN EP Can ONNX Sheet.1011 ONNX Rectangle.1010 Input Input Can.1011 ctx Sheet.1014 ctx Sheet.1015 model.onnx with context.bin model.onnx with context.bin Sheet.1016 Sheet.1017 Sheet.1018 Sheet.1019 Dynamic connector Sheet.1021 Dynamic connector.1022 Dynamic connector.1023 Sheet.1024 Sheet.1025 Sheet.1026 Sheet.1027 ### Use a Qualcomm AI Runtime SDK model Use the Qualcomm AI Runtime SDK (QAIRT) for hardware-specific optimization by converting and quantizing models into a QNN representation. QAIRT generates a SoC-optimized context and packages it as an ONNX file, commonly known as a `ctx.onnx` model. This model is tailored for a specific Qualcomm SoC and can be directly executed using the QNN EP in the ONNX runtime. Page-1 Rectangle.10 Host (Linux Ubuntu 22.04) Host (Linux Ubuntu 22.04) Rectangle.16 Conversion Conversion Rectangle.17 Quantization Quantization Rectangle.18 Rectangle.1010 Input Input Rectangle.1033 Sheet.1034 QAIRT QAIRT Rectangle.1035 Sheet.1036 ONNX ONNX Rectangle.1039 model.onnx model.onnx Rectangle.1040 model.bin model.bin Rectangle.1041 model.cpp model.cpp Rectangle.1042 Compilation Compilation Rectangle.1043 model.so model.so Rectangle.1044 conversion conversion Rectangle.1045 model.bin model.bin Rectangle.1046 conversion conversion Rectangle.1047 model.onnx model.onnx Sheet.1048 Sheet.1049 Rectangle.1050 model_net.json model_net.json Rectangle.1051 backend_extension.json backend_extension.json Rectangle.1052 libQNNHtp.so libQNNHtp.so Sheet.1053 Sheet.1054 Sheet.1056 Sheet.1057 Sheet.1058 Sheet.1059 Sheet.1060 Sheet.1061 Sheet.1062 Sheet.1063 Dynamic connector Sheet.1065 qnn-onnx-converter.py qnn-onnx-converter.py Sheet.1066 qnn-model-lib-generator.py qnn-model-lib-generator.py Sheet.1067 qnn-context-binary-generator.py qnn-context-binary-generator.py Sheet.1068 gen_qnn_ctx_onnx_model.py gen_qnn_ctx_onnx_model.py ### Deployment workflow Use this workflow to build ORT with the QNN EP, validate the build, and deploy the model and runtime artifacts to the target device for NPU inference. Page-1 Rectangle.10 Rectangle.18 Yocto Linux Yocto Linux Rectangle.19 libQnnHtp.so libQnnHtp.so Rectangle.20 model.onnx (qdq / qnn_ctx) model.onnx (qdq / qnn_ctx) Rectangle.1010 Input Input Rectangle.1031 Output Output Rectangle.1032 Ubuntu Ubuntu Rectangle.1033 Output Output Sheet.1039 IQ-9 / RB3 Gen 2 HTP IQ-9 / RB3 Gen 2 HTP Sheet.1040 Qualcomm IM SDK docker container Qualcomm IM SDK docker container Sheet.1041 run_qnn_ctx.py / run_qdq_model.py run_qnn_ctx.py /run_qdq_model.py Sheet.1042 Native Native Sheet.1043 Run C++ sample application Run C++ sample application Sheet.1045 Native Native Rectangle.1038 Deployment Deployment Sheet.1044 run_qnn_ctx.py / run_qdq_model.py run_qnn_ctx.py /run_qdq_model.py Sheet.1046 Run C++ sample application Run C++ sample application Sheet.1048 Sheet.1049 Sheet.1050 Sheet.1051 Sheet.1052 Sheet.1053 Last Published: Jul 13, 2026 [Previous Topic Run a LiteRT model on NPU](https://docs.qualcomm.com/bundle/publicresource/80-70030-15B/topics/run-a-litert-model-using-delegate.md) [Next Topic Prepare ONNX models](https://docs.qualcomm.com/bundle/publicresource/80-70030-15B/topics/onnx-prepare-model.md)