# Writing QNN HTP Op Package

Writing QNN HTP op packages can be divided into two major sections:

- writing op package interface
- writing op implementations and optimization rules

In the QNN HTP op package example located at examples/OpPackage/HTP/ in QNN SDK,
these sections are divided into separate source files:

- One op package interface file is needed per package: ExampleOpPackageInterface.cpp
- Any number of individual op implementation files can be included:
ExampleOpPackageRelu.cpp, ExampleOpPackageSoftmax.cpp, ExampleOpPackageMaxPool.cpp

This document explains the general rules and approaches to compose QNN HTP op packages,
and it focuses on op package interface section, and it uses ExampleOpPackageInterface.cpp
as the example file. For descriptions on how to write op implementations and optimization
rules, please refer to other documents located at the same locations. For descriptions
on ExampleOpPackageRelu.cpp, please refer to relu\_example.html.

ExampleOpPackageInterface.cpp can be used as a template for any QNN HTP op package.

## API’s in QnnOpPackage.h

To write a QNN op package, please exam QnnOpPackage.h located at include/ directory in
QNN SDK first for data structure and API definitions. ExampleOpPackageInterface.cpp
defines HTP specific op package structures and QNN op package API functions listed in
QnnOpPackage.h. Currently HTP only supports and requires the following API’s:

typedef Qnn_ErrorHandle_t (*QnnOpPackage_InterfaceProvider_t)(QnnOpPackage_Interface_t* interface);
    
    typedef Qnn_ErrorHandle_t (*QnnOpPackage_InitFn_t)(QnnOpPackage_GlobalInfrastructure_t infrastructure);
    
    typedef Qnn_ErrorHandle_t (*QnnOpPackage_GetInfoFn_t)(const QnnOpPackage_Info_t** info);
    
    typedef Qnn_ErrorHandle_t (*QnnOpPackage_LogInitializeFn_t)(QnnLog_Callback_t callback, QnnLog_Level_t maxLogLevel);
    
    typedef Qnn_ErrorHandle_t (*QnnOpPackage_LogSetLevelFn_t)(QnnLog_Level_t maxLogLevel);
    
    typedef Qnn_ErrorHandle_t (*QnnOpPackage_LogTerminateFn_t)(void);
    
    typedef Qnn_ErrorHandle_t (*QnnOpPackage_TerminateFn_t)();
    Copy to clipboard

> 
> 
> **Aside**: Following structs listed in QnnOpPackage.h are not
> defined or used in HTP backend
> :
> `QnnOpPackage_GlobalInfrastructure_t`,
> `QnnOpPackage_GraphInfrastructure_t`, `QnnOpPackage_Kernel_t`,
> `QnnOpPackage_Node_t`, `QnnOpPackage_Optimization_t`

## HTP Headers

HTP core provides a list of headers which HTP op packages can and shall use to implement
ops, to define optimization rules, to register ops and optimization rules, to specify op
parameter orders and more. These headers can be found in QNN SDK under include/HTP/core/,
and more info can be found in htp\_core\_headers.html. Please directly include the following
headers in HTP op package source files.

#include "QnnOpPackage.h"
    #include "constraints.h"
    #include "op_package_feature_support.h"
    #include "op_register_ext.h"
    #include "optimize.h"
    #include "HTP/core/simple_reg.h"
    #include "HTP/core/unique_types.h"
    Copy to clipboard

## HTP Macros

HTP op package shall utilize a series of macros defined in HTP core headers to register
ops, to define and register optimization rules, to list op parameter orders, to list
axis parameters, and to list per-channel scale ops.

**Op registration**

Besides defining op execution functions for each op package op, HTP also requires op
packages to register the ops.

/*
     * op initialization
     *
     * needs to be global in the package
     * one initialization per package before any op definitions
     */
    INIT_PACKAGE_OP_DEF()
    
    /*
     * op registration
     *
     * unified core init function containing ops/opts registration
     * needs to be global in the package
     */
    INIT_PKG_CORE_INIT_FUNC()
    
    /*
     * op definition
     *
     * shall be used in individual op implementation source files
     * needs to be global in the package
     * one definition per op
     * please refer to implementing_ops.html for more descriptions
     */
    DEF_PACKAGE_OP(F,OP)
    DEF_PACKAGE_OP_AND_COST_AND_FLAGS(F,OP,COST,...)
    DEF_PACKAGE_OP_AND_COST_F_AND_FLAGS(F,OP,COST_F,...)
    Copy to clipboard

**Set tensor properties**

Define op tensor properties to centralize the
decision-making on the Layout and Memory Placement of our tensors

DEF_TENSOR_PROPERTIES(Op("Op", "in", "in2"),
              Flat("*", "in2"),
              MainMemory("..."))
    Copy to clipboard

### Constraint terms:

- Flat: flat layout
- Crouton: crouton layout
- Tcm: in TCM
- MainMemory: in main memory.

Specify the requirements for operators
**Optimization rule registration**

Op packages can define optimization rules which alter HTP graphs in desired ways.
For more info, please refer to optimization\_grammar.html.

/*
     * optimization initialization
     *
     * needs to be global in the package
     * one initialization per package before any optimization definitions
     */
    INIT_PACKAGE_OPTIMIZATION_DEF()
    
    /*
     * optimization definition
     *
     * shall be used in individual op implementation source files
     * needs to be global in the package
     * one definition per optimization
     * please refer to implementing_ops.html for more descriptions
     */
    DEF_PACKAGE_OPTIMIZATION(PRIORITY,MATCHCODE,CONSTRAINTCODE,REPLACECODE)
    Copy to clipboard

**Op parameter order**

Op packages can specify parameter orders and define default parameter values for
any ops defined in the current package. This is optional in op packages, if no
parameter order has been specified for an op, then the parameter order depends
on the order provided at QnnGraph\_addNode.

/*
     * op parameter order initialization
     *
     * needs to be global in the package
     * one initialization per package before any op parameter order definitions
     */
    INIT_PACKAGE_PARAM_ORDER_DEF()
    
    /*
     * op parameter order registration
     *
     * needs to placed in op package initialization function
     * registers all defined op parameter orders in the package
     */
    REGISTER_PACKAGE_PARAM_ORDERS()
    
    /*
     * op parameter order definition
     *
     * shall be used in individual op implementation source files
     * needs to be global in the package
     * one definition per op, and this is optional
     * please refer to implementing_ops.html for more descriptions
     */
    DEF_PACKAGE_PARAM_ORDER(OP,PARAM1,MANDATORY1,DEFAULT1,PARAM2,MANDATORY2,DEFAULT2...)
    Copy to clipboard

**Axis parameter (optional)**

HTP currently only supports 4-dimensional tensors, however, QNN supports tensors of
any dimensions, so HTP backend does tensor dimension backfilling. Due to this discrepancy,
any op parameters that refer to axes need to be adjusted in HTP backend. HTP allows
op packages to list the op parameter names which should be identified as axis parameter,
and HTP backend does auto-adjustment to fit axis into 4-dimensional tensors. This is
optional in op packages.

/*
     * axis parameter name list
     *
     * optional
     * needs to be global in the package
     * one list per package
     * e.g. LIST_PACKAGE_AXIS_PARAMS("Axis", "AXIS", "axis")
     */
    LIST_PACKAGE_AXIS_PARAMS(...)
    
    /*
     * op axis parameter name registration
     *
     * optional
     * uses with LIST_PACKAGE_AXIS_PARAMS(...)
     * needs to placed in op package initialization function
     * registers all axis parameter names in the package
     */
    REGISTER_PACKAGE_AXIS_PARAMS()
    Copy to clipboard

**Per-channel scale op (optional)**

HTP supports usage of per-axis quantized tensors in op packages, however, the support
limits to per-channel scale tensors only. That means the axis used in per-axis quantized
tensors can only be along the last dimension, and the quantization offset value can only
be zero. In QNN tensors, per-axis quantization info is packed into the tensors if
QNN\_QUANTIZATION\_ENCODING\_AXIS\_SCALE\_OFFSET is set as the quantization encoding type.
In HTP backend, per-channel scale values come in as separate HTP tensors, in addition to
the HTP tensors containing data values.

Op packages shall list the op names which support per-channel scale tensors using the
macros shown below. For ops listed using the macros below, when any of the input tensors,
parameters, and output tensors used in QnnGraph\_addNode has QNN\_QUANTIZATION\_ENCODING\_AXIS\_SCALE\_OFFSET
quantization encoding type, HTP backend passes additional HTP tensors to op execution
functions to represent per-channel scale values. For regular ops, HTP tensors passed into
op execution functions are outputs, inputs, parameters. For per-channel scale ops, HTP
tensors passed into op execution functions are outputs, inputs, parameters, output per-
channel scale values, input per-channel scale values, parameter per-channel scale values.
HTP fills default per-channel scale value of 1 for any non-QNN\_QUANTIZATION\_ENCODING\_AXIS\_SCALE\_OFFSET
tensors in the above case. Per-channel scale values of an output or an input or a parameter
in op implementation functions can be accessed by <cite>tensor(0,0,0,d)</cite>, given <cite>tensor</cite>
being the corresponding per-channel scale value tensor, and given d being the channel
of interest.

/*
     * per-channel quantized op name list
     *
     * optional
     * needs to be global in the package
     * one list per package
     */
     LIST_PACKAGE_PER_CHANNEL_QUANTIZED_OPS(...)
    
    /*
     * per-channel scale op name registration
     *
     * optional
     * uses with LIST_PACKAGE_PER_CHANNEL_QUANTIZED_OPS(...)
     * needs to placed in op package initialization function
     * registers all per-channel scale op names in the package
     */
     REGISTER_PACKAGE_PER_CHANNEL_QUANTIZED_OPS()
    Copy to clipboard

## Compilation

Name of a QNN HTP op package is independent from source code, and the name must be passed
as a compilation flag using the following format:

`-DTHIS_PKG_NAME=YOUR_PACKAGE_NAME`

Current QNN HTP op packages are required to be re-compiled after every QNN SDK release.

For example makefile, please refer to makefile located at examples/OpPackage/HTP/ in
QNN SDK.

Last Published: Jun 04, 2026

[Previous Topic
Tensors and Memory Layout](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/tensors_and_memory_layout.md) [Next Topic
General OpPackage Central Migration Guidance](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/central_migration_guidance.md)