ai-arch

# Architecture The Qualcomm AI offering consists of hardware accelerators and AI SDKs to harness the power of hardware. ## AI hardware accelerators AI workloads can be accelerated on multiple hardware cores: - Qualcomm^®^ Hexagon™ Tensor Processor (HTP) - Also known as NPU/DSP/HMX, suitable to execute AI workloads with low-power and high-performance. For optimized performance, pre-trained models need be quantized to one of the supported precisions. - Qualcomm^®^ Adreno™ GPU - Suitable to execute AI workloads with medium-power, and medium-performance. AI workloads are accelerated with OpenCL kernels. The GPU can also be used to accelerate model pre/post-processing. - Qualcomm^®^ Kryo™ CPU - AI inferencing on CPU can be used to benchmark model accuracy/performance against other hardware accelerators. The CPU can also be used to run model pre/post processing. ## AI software stack AI stack contains SDKs to harness the power of AI hardware accelerators. Developers can use the stack of their choice to deploy AI workloads. Pre-trained models (with the exception of TFLite models) need to be converted to an executable format with the selected AI Stack SDK before running them. Note that TFLite Delegate allows developers to directly run TFLite models. - [Qualcomm Neural Processing Engine (SNPE)](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-2) > > > An all-in-one SDK that provides C, C++, and Java APIs to support heterogenous computing, system-level configurations, and direct AI workloads to all accelerator cores. > Provides developers with flexibility, including inter-core collaboration support and other advanced features. - [Qualcomm AI Engine Direct (QNN)](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50) > > > Lower-level, highly customizable unified APIs that speed up AI models on all AI accelerator cores with individual libraries. > Can be used directly to target a specific accelerator core or delegate workloads from popular runtimes including Qualcomm Neural Processing Engine SDK, TensorFlow Lite, and ONNX runtime. > Low-level SDK provides more functionality and debugging abilities. - [AI Model Efficiency Toolkit (AIMET)](https://quic.github.io/aimet-pages/releases/latest/index.html) > > > Open-source library to optimize (compressing and quantizing) trained neural network models. > This is a complex SDK designed to generate optimized quantized models. It’s intended only for advanced developers. Last Published: Mar 27, 2025 [Previous Topic Overview](https://docs.qualcomm.com/bundle/publicresource/80-70018-15/topics/overview.md) [Next Topic APIs](https://docs.qualcomm.com/bundle/publicresource/80-70018-15/topics/interfaces.md)