# QNN delegate
Source: [https://docs.qualcomm.com/doc/80-70014-54/topic/qnn-delegate.html](https://docs.qualcomm.com/doc/80-70014-54/topic/qnn-delegate.html)
The QNN delegate is a proprietary delegate for vendor-specific hardware acceleration
to accelerate TensorFlow Lite models. The QNN delegate is designed based on the TensorFlow
Lite's [external delegate interface](https://www.tensorflow.org/lite/performance/implementing_delegate). You can use the QNN
delegate to offload parts or the entire TensorFlow Lite model to specialized Qualcomm
hardware, such as the Adreno GPU and Hexagon Tensor Processor.
The QNN delegate improves the performance of model execution and power efficiency by
decreasing the CPU workload.
The QNN delegate also uses the existing Qualcomm AI Engine Direct APIs and available
backends to accelerate models. For more information, see [Qualcomm AI Engine Direct](bundle/publicresource/topics/80-63442-50).
The QNN delegate can execute models in 32‑bit floating-point precision and INT8 precision
on the available hardware.
You can build applications using the following interfaces:
- Qualcomm AI Engine Direct delegate interface
- TensorFlow Lite External delegate interface
You can access both the interfaces when using a standalone TensorFlow Lite application.
However, if you deploy your TensorFlow Lite models using the IM SDK, the qtimltflite
GStreamer plug-in for Qualcomm TensorFlow Lite Runtime uses the external delegate
interface of TensorFlow Lite. For more information, see [Leverage external delegate](https://www.tensorflow.org/lite/performance/implementing_delegate#option_2_leverage_external_delegate).
The following figure shows the directory structure of QNN delegate libraries from the
Qualcomm AI Engine Direct SDK:
Figure : QNN delegate directory structure
## Qualcomm AI Engine Direct delegate interface (QNN delegate)
The QNN delegate provides the `QnnTFLiteDelegate.h` header as an
interface. You can include this header as part of your application before linking it
to the QNN delegate library.
For RB3 Gen 2, you can find a compatible QNN delegate library and QNN libraries
placed in the `aarch64-oe-linux-gcc11.2 cross-compiler` toolchain
triplet directory.
Table : QNN delegate acceleration support
| Backend name | Backend description | Target and library names | Library description |
| --- | --- | --- | --- |
| CPU | Backend for Arm CPU acceleration | | `libQnnCpu.so`: CPU backend library |
| GPU | Backend for the Adreno GPU hardware accelerator | | `libQnnGpu.so`: GPU backend library |
| Hexagon Tensor Processor | Backend for the Hexagon Tensor Processor hardware
accelerator |
aarch64-oe-linux-gcc11.2
libQnnHtp.so
libQnnHtpPrepare.so
libQnnHtpV68Stub.so
hexagon-v68
|
libQnnHtp.so: Library used for
communication between the Arm CPU and Hexagon Tensor
Processor.
libQnnHtpPrepare.so: Hexagon Tensor
Processor library running on the CPU side. This library is
loaded to prepare the graph and optimize the execution graph
at runtime.
libQnnHtpV68Stub.so: Hexagon Tensor
Processor proxy library on the CPU side, communicating with
the Skel library loaded on the Hexagon Tensor
Processor.
libQnnHtpV68Skel.so: Hexagon Tensor
Processor native library with optimized kernel/operator
implementation for Hexagon Tensor Processor v68.
|
## TensorFlow Lite external delegate interface
In an external delegate interface, the application to run TensorFlow Lite models must
load the `libQnnTFLiteDelegate.so` delegate library of QNN. The C/C++
application and the `libQnnTFLiteDelegate.so` delegate library have
no dependency on each other. Therefore, if the delegate changes, you do not have to
recompile the application.
In addition to the TensorFlow Lite Android C API, integrate the following into the
C/C++ Android application in the same way as the TensorFlow Lite Android C API:
- The external\_delegate.h header file
- The libexternal\_delegate.so shared library
The QNN delegate offers acceleration on the Hexagon Tensor Processor, GPU, and CPU.
To customize where and how to run models using the QNN delegate when using the
external delegate interface, you must provide additional external delegate options.
For instructions, see [External delegate options for QNN delegate](https://docs.qualcomm.com/doc/80-70014-54/topic/external-delegate-options-for-qnn-delegate.html).
**Parent Topic:** [Delegates](https://docs.qualcomm.com/doc/80-70014-54/topic/delegates.html)
Last Published: Jul 12, 2024
[Previous Topic
GPU delegate](https://docs.qualcomm.com/bundle/publicresource/80-70014-54/topics/gpu-delegate.md) [Next Topic
TensorFlow Lite developer workflow](https://docs.qualcomm.com/bundle/publicresource/80-70014-54/topics/tensorflow-lite-developer-workflow.md)