# Usage guidelines

## Use QnnInterface to access QNN APIs

The QNN Interface is an abstraction that combines all component APIs and provides a single unified
interface to the user. This provides a major convenience to users to operate with a fixed set of
API hooks through the interface, instead of relying on dynamic symbol lookup using `dlsym()` calls.

The QNN Interface API is made available to users using what are referred to as ‘API providers’.
API providers are based on the publisher-subscriber paradigm with which backends publish
access points to APIs they support, and clients subscribe to them dynamically.

[QnnInterface\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/structQnnInterface__t.html#exhale-struct-structqnninterface-t) demonstrates the central unit
that holds information about the provider (its name, the backend that publishes it, and
the collection of APIs it supports) as function pointers.

The client code should invoke [QnnInterface\_getProviders()](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnInterface_8h_1ac794dff8e62c4656225be0cc4a282b76.html#exhale-function-qnninterface-8h-1ac794dff8e62c4656225be0cc4a282b76)
to query all providers that are available on the system, and access individual
component APIs through them.

## Use initializer macros for struct data types

Initializer macros defined for `struct` data types in API headers have the recommended default
values initialized for various fields. It is recommended that clients use these macros when
declaring data structure variables to avoid depending on compiler defaults, which may vary by
platform or architecture.

The following code snippet shows a sample usage of initializing [Qnn\_Tensor\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/structQnn__Tensor__t.html#exhale-struct-structqnn-tensor-t)
using the [QNN\_TENSOR\_INIT](https://docs.qualcomm.com/doc/80-63442-50/topic/define_QnnTypes_8h_1a101cd9953298300882fbb6f7050c7335.html#exhale-define-qnntypes-8h-1a101cd9953298300882fbb6f7050c7335) init macro.

1// Default initialization using init macro
     2Qnn_Tensor_t tensor          = QNN_TENSOR_INIT;
     3
     4// Update fields with context-specific values
     5tensor.version               = QNN_TENSOR_VERSION_1;
     6tensor.v1.name               = "tensor";
     7tensor.v1.type               = QNN_TENSOR_TYPE_APP_WRITE;
     8tensor.v1.dataType           = QNN_DATATYPE_FLOAT_32;
     9tensor.v1.rank               = rank;
    10tensor.v1.dimensions         = dimensions;
    11tensor.v1.clientBuf.data     = data;
    12tensor.v1.clientBuf.dataSize = numElements * sizeof(float);
    Copy to clipboard

## Read error codes from QNN API

QNN APIs return error codes bundled into
[Qnn\_ErrorHandle\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/typedef_QnnCommon_8h_1a8e21e7c6a7471103b59098ef81c28570.html#exhale-typedef-qnncommon-8h-1a8e21e7c6a7471103b59098ef81c28570).
However, users must note from the description of the field that the actual error codes
are to be read out from the least significant 16 bits of this 64-bit handle.

Users may use the
[QNN\_GET\_ERROR\_CODE](https://docs.qualcomm.com/doc/80-63442-50/topic/define_QnnCommon_8h_1a858eb9083d3c3ad2f65dbe687cddea56.html#exhale-define-qnncommon-8h-1a858eb9083d3c3ad2f65dbe687cddea56) macro
to extract the return code. It is recommended that users always do this when handling return
values from QNN APIs. Successful invocation of a QNN API will return
[QNN\_SUCCESS](https://docs.qualcomm.com/doc/80-63442-50/topic/define_QnnCommon_8h_1af3d413884b927f4afd7a1222ab0f6620.html#exhale-define-qnncommon-8h-1af3d413884b927f4afd7a1222ab0f6620)
and no additional processing is required in this case.

## Multi-Threading

QNN APIs invoked on the same handle (e.g.,
[Qnn\_GraphHandle\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/typedef_QnnCommon_8h_1aac60af5d8bde388852ae15bd89fbcd3c.html#exhale-typedef-qnncommon-8h-1aac60af5d8bde388852ae15bd89fbcd3c)), a parent handle (e.g., a
[Qnn\_ContextHandle\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/typedef_QnnCommon_8h_1ae2751b09a1186e90f79e7b89d1903c8f.html#exhale-typedef-qnncommon-8h-1ae2751b09a1186e90f79e7b89d1903c8f) is a parent of a
[Qnn\_GraphHandle\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/typedef_QnnCommon_8h_1aac60af5d8bde388852ae15bd89fbcd3c.html#exhale-typedef-qnncommon-8h-1aac60af5d8bde388852ae15bd89fbcd3c)), or a dependents handle (e.g.,
a [Qnn\_ContextHandle\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/typedef_QnnCommon_8h_1ae2751b09a1186e90f79e7b89d1903c8f.html#exhale-typedef-qnncommon-8h-1ae2751b09a1186e90f79e7b89d1903c8f) depends upon a
[Qnn\_DeviceHandle\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/typedef_QnnCommon_8h_1a5ef1ac85057d5ffe9a7a22e5682f0cb3.html#exhale-typedef-qnncommon-8h-1a5ef1ac85057d5ffe9a7a22e5682f0cb3)) are not thread safe.

## Allocate external memory to register with a backend

QNN APIs defined in [QnnMem.h](https://docs.qualcomm.com/doc/80-63442-50/topic/api-rst_file_include_QNN_QnnMem_h.html##file-include-qnn-qnnmem-h) provide a mechanism to register
externally allocated memory with a backend. Data passed via shared memory can be read from directly,
allowing for zero-copy inference, unlike data passed via a raw pointer, which generally may be
copied before graph execution.

Currently, ION is the only external memory type supported by QNN. ION is only available on Android
platforms. One of the methods to allocate ION memory is via the RPCMem framework provided by the
Hexagon SDK.

Specifically, the following RPCMem APIs are required:

> 
> 
> - `rpcmem_alloc()` – to allocate ION memory.
> - `rpcmem_to_fd()` – to obtain a file descriptor that refers to this allocated memory, which can be
> registered with a backend via [QnnMem\_register()](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnMem_8h_1af084b3b2a3c20636c083aeff0aa74953.html#exhale-function-qnnmem-8h-1af084b3b2a3c20636c083aeff0aa74953).
> - `rpcmem_free()` – to free the allocated memory.

Refer to the Hexagon SDK documentation for more information about RPCMem.

## Qnn\_TensorV2\_t

The [Qnn\_TensorV2\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/structQnn__TensorV2__t.html#exhale-struct-structqnn-tensorv2-t) data structure introduces three new features to the
[Qnn\_Tensor\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/structQnn__Tensor__t.html#exhale-struct-structqnn-tensor-t) structure: sparsity, dynamic shape, and tensor production
notification.

Warning

Note that in QNN SDK 2.20.0, there is no backend or tool support added for Qnn\_TensorV2\_t features. Support
will begin to be added in QNN SDK 2.21.0 and subsequent releases.

### Sparsity

Backend support for the tensor sparsity feature can be queried via
[QnnProperty\_hasCapability](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnProperty_8h_1a025033c14ee4522582f1e89d4086f8ae.html#exhale-function-qnnproperty-8h-1a025033c14ee4522582f1e89d4086f8ae) using the
QNN\_PROPERTY\_TENSOR\_SUPPORT\_SPARSITY capability.

Tensor sparsity can be expressed via [Qnn\_TensorV2\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/structQnn__TensorV2__t.html#exhale-struct-structqnn-tensorv2-t) by setting the
Qnn\_TensorV2\_t::dataFormat field to QNN\_TENSOR\_DATA\_FORMAT\_SPARSE along with setting Qnn\_TensorV2\_t::sparseParams. The
[Qnn\_SparseParams\_t](https://docs.qualcomm.com/doc/80-63442-50/topic/structQnn__SparseParams__t.html#exhale-struct-structqnn-sparseparams-t) data structure is extensible, but only supports a
hybrid COO sparse format. This sparse format allows for partially sparse tensors and more details will follow in
subsequent releases.

New operations have been added to support networks taking advantage of tensor sparsity: CreateSparse, GetSparseIndices,
GetSparseValues, and SparseToDense. Other operations will be updated to support sparsity as needed.

### Dynamic dimensions

Backend support for the tensor dynamic shape feature can be queried via
[QnnProperty\_hasCapability](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnProperty_8h_1a025033c14ee4522582f1e89d4086f8ae.html#exhale-function-qnnproperty-8h-1a025033c14ee4522582f1e89d4086f8ae) using the
QNN\_PROPERTY\_TENSOR\_SUPPORT\_DYNAMIC\_DIMENSIONS capability.

Dynamic shape allows for tensors to change shape from inference to inference. A tensor can be created with dynamic shape
by setting the Qnn\_TensorV2\_t::isDynamicDimensions field to an array of uint8\_t values. If a value is non-zero, it
indicates that the corresponding dimension has a dynamic size. If a value is zero, the corresponding dimension is
statically sized. If the Qnn\_TensorV2\_t::isDynamicDimensions field is set to NULL, it indicates the entire tensor has a
static shape. When a tensor is created by calling
[QnnTensor\_createGraphTensor](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnTensor_8h_1acf98aa32e9fc9db88e2250f75c7e21ac.html#exhale-function-qnntensor-8h-1acf98aa32e9fc9db88e2250f75c7e21ac) or
[QnnTensor\_createContextTensor](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnTensor_8h_1ae4cb118c85d47fcc5f68dbe4f095cfec.html#exhale-function-qnntensor-8h-1ae4cb118c85d47fcc5f68dbe4f095cfec), the
Qnn\_TensorV2\_t::dimensions field is interpreted as the maximum size the dimension can have. When the tensor is provided
during a call to [QnnGraph\_execute](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnGraph_8h_1a3ea05f42a9295f9a74a2e3a0cdd64228.html#exhale-function-qnngraph-8h-1a3ea05f42a9295f9a74a2e3a0cdd64228), the
Qnn\_TensorV2\_t::dimensions field is interpreted as the actual tensor shape. For dynamically shaped graph input/output
tensors, the  Qnn\_TensorV2\_t::dimensions field must be populated with actual dimensions. If the maximum dimensions for
a tensor were exceeded during a call to
[QnnGraph\_execute](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnGraph_8h_1a3ea05f42a9295f9a74a2e3a0cdd64228.html#exhale-function-qnngraph-8h-1a3ea05f42a9295f9a74a2e3a0cdd64228), the
QNN\_GRAPH\_ERROR\_DYNAMIC\_TENSOR\_SHAPE error code will be returned. In this scenario, it is possible that the tensor data
is not valid.

### Early termination

Backend support for the early termination and output tensor production feature can be queried via
[QnnProperty\_hasCapability](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnProperty_8h_1a025033c14ee4522582f1e89d4086f8ae.html#exhale-function-qnnproperty-8h-1a025033c14ee4522582f1e89d4086f8ae) using the
QNN\_PROPERTY\_GRAPH\_SUPPORT\_EARLY\_TERMINATION capability.

An operation may have a capability to terminate graph execution early. If a call to
[QnnGraph\_execute](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnGraph_8h_1a3ea05f42a9295f9a74a2e3a0cdd64228.html#exhale-function-qnngraph-8h-1a3ea05f42a9295f9a74a2e3a0cdd64228) terminates early, the
QNN\_GRAPH\_ERROR\_EARLY\_TERMINATION error code will be returned. If this error code is detected, it is possible that not
all graph output tensors were produced. If the Qnn\_TensorV2\_t::isProduced field is non-zero, the graph output
tensor data was produced and can be consumed.  If the Qnn\_TensorV2\_t::isProduced field is zero, the tensor data was not
produced and is not valid.

## Control graph priority

Backend support for the graph priority control feature can be queried via
[QnnProperty\_hasCapability](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnProperty_8h_1a025033c14ee4522582f1e89d4086f8ae.html#exhale-function-qnnproperty-8h-1a025033c14ee4522582f1e89d4086f8ae) using the
QNN\_PROPERTY\_GRAPH\_SUPPORT\_PRIORITY\_CONTROL capability.

When a backend supports graph priority control, clients can assign priority configuration during
graph creation (using [QnnGraph\_create](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnGraph_8h_1acef7b5c50463d8b02bb4abd769ae5ea9.html#exhale-function-qnngraph-8h-1acef7b5c50463d8b02bb4abd769ae5ea9))
or modify the priority of an existing graph (using [QnnGraph\_setConfig](https://docs.qualcomm.com/doc/80-63442-50/topic/function_QnnGraph_8h_1ac0ba7ace966622bc3967a29f334b157e.html#exhale-function-qnngraph-8h-1ac0ba7ace966622bc3967a29f334b157e)).
In general the priority configuration controls how inference jobs are scheduled and executed. However, the exact results
vary for different backends. See the backend-specific documentation for more information about priority and its related behavior.

Last Published: Oct 10, 2025

[Previous Topic
Backend API specialization](https://docs.qualcomm.com/bundle/publicresource/80-63442-50/topics/api_overview.md) [Next Topic
QNN API Revision History](https://docs.qualcomm.com/bundle/publicresource/80-63442-50/topics/api_version_history.md)