# QNN Operation Definitions This document defines operations supported by QNN. Per backend specific information is referenced as needed. ## ArgbToRgb Transform ARGB or RGBA to RGB. Refer to input\_order and reverse\_output below for control of the input/output order. ### Inputs #### in[0] Input tensor - Mandatory: true - Data type: backend specific - Shape: [b,h,w,4] ### Parameters #### input\_order Controls the order of the input tensor. If QNN\_OP\_ARGB\_TO\_RGB\_INPUT\_ORDER is QNN\_OP\_ARGB\_TO\_RGB\_INPUT\_ORDER\_ARGB the input order is ARGB; if QNN\_OP\_ARGB\_TO\_RGB\_INPUT\_ORDER\_RGBA is selected then the input order is RGBA. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - ARGB = 0, > - RGBA = 1 #### reverse\_output Controls the order of the output tensor. Set to false the order of the input tensor is maintained in the output tensor. Set to true, the order of the last 3 channels of the input tensor is reversed in the output tensor. - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar ### Outputs #### out[0] Output tensor - Mandatory: true - Data type: backend specific - Shape: [b,h,w,3] - Constraints: > > > - Datatype: Same datatype as in[0] ## Argmax Returns the index of the largest element along an axis. References: - ONNX: [ops::ArgMax](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ArgMax) - Android NDK NeuralNetworks: [ANEURALNETWORKS_ARGMAX](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0af78048ff71a1e79bbc6034d30cad7cbc) - QNN: [Argmin](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#argmin) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: n-dimensional - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axis Axis on which to reduce across. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be in range [0, n-1] #### keep\_dims If true, the resulting tensor has the same number of dimensions as the input tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32, backend specific - Shape: m-dimensional, where m = n if *keep\_dims* is true and m = n - 1 otherwise. ## Argmin Returns the index of the smallest element along an axis. References: - ONNX: [ops::ArgMin](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ArgMin) - Android NDK NeuralNetworks: [ANEURALNETWORKS_ARGMIN](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a89be86ff36e83254e9b3d0954e7e0729) - QNN: [Argmax](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#argmax) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: n-dimensional - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axis Axis on which to reduce across. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be in range [0, n-1] #### keep\_dims If true, the resulting tensor has the same number of dimensions as the input tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32, backend specific - Shape: m-dimensional, where m = n if *keep\_dims* is true and m = n - 1 otherwise. ## AxisAlignedBboxTransform Transform axis-aligned bounding box proposals into refined bounding boxes using bounding box regression deltas for each class. Notes: - Bounding boxes are aligned to the image coordinate system (i.e. not rotated). - Axis-aligned bounding boxes are defined by the upper-left corner (x1, y1) and lower-right corner (x2, y2). - Valid bounding boxes are such that x1 <= x2 and y1 <= y2. - Resulting bounding boxes are clipped against the edges of the image. - The number of regions of interest (num\_rois) is in the range [0, N] where N is the maximum set by the underlying tensor. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_AXIS_ALIGNED_BBOX_TRANSFORM](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0afd7603dd54060e6a52f5861674448528) - QNN: [BboxTransform](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#bboxtransform) ### Inputs #### in[0] Bounding box proposal locations - Mandatory: true - Data type: backend specific - Shape: [num\_rois, 4] each line with format [x1, y1, x2, y2] #### in[1] Bounding box deltas for each region of interest and each class Bounding box deltas are organized in the following order [dx, dy, dw, dh] where: - dx and dy are the relative correction factors for the center position of the bounding box - dw and dh are the log-scale relative correction factors for the width and height the bounding box - Mandatory: true - Data type: backend specific - Shape: [num\_rois, num\_classes \* 4] #### in[2] Batch index of each bounding box. Boxes with the same batch index are grouped together. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [num\_rois] #### in[3] Specifies image size. Image size is the same for all images in the batch. - Mandatory: true - Data type: backend specific - Shape: [batches, 2] with format [image\_height, image\_width] per batch ### Parameters #### weights Weights applied to each of the bounding boxes deltas in in[1]. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: [4] with format [wx, wy, ww, wh] - Default: [1.0, 1.0, 1.0, 1.0] ### Outputs #### out[0] Coordinates of refined bounding boxes - Mandatory: true - Data type: backend specific - Shape: [num\_rois, num\_classes \* 4] with format [x1, y1, x2, y2] - Constraints: > > > - Datatype: Same datatype as in[0] #### out[1] batch\_splits : Specifies the number of RoIs/boxes belonging to the corresponding image in batch. Note that the sum of values should add up to a total of num\_rois. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [batches] ## Batchnorm Normalizes the activations of the previous layer at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1. See Batchnorm backend definition for supported datatypes and constraints per backend. References: - TensorFlow: [Batch Normalization](https://www.tensorflow.org/api_docs/python/tf/nn/batch_normalization) - ONNX: [BatchNormalization](https://github.com/onnx/onnx/blob/master/docs/Operators.md#BatchNormalization) ### Inputs #### in[0] Input tensor from the previous operator - Mandatory: true - Data type: backend specific - Shape: n-dimensional, note that the last dimension in the input is the channel, […,channel]. - Constraints: > > > - Shape: Rank > 0 #### in[1] Weights - Mandatory: true - Data type: backend specific - Shape: [channel] #### in[2] Biases - Mandatory: false - Data type: backend specific - Shape: [channel] - Default: {0,..,0} ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: n-dimensional - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] > - Must have same data format as in[0] (e.g. both sparse or both dense) ## BatchPermutation Generates a batch permutation of the input in[0]. The output out[0] may have the same shape as in[0] with the exception of the batch dimension which is the shape of in[1]. Data is re-ordered according to the indices provided. Example of batch permutation on a 3-D tensor with batch size 4: input = [ [[1, 5], [3, 4]], [[4, 3], [5, 2]], [[2, 2], [6, 0]], [[0, 0], [1, 2]] ] indices = [2, 0, 1, 3] output = [ [[2, 2], [6, 0]], [[1, 5], [3, 4]], [[4, 3], [5, 2]], [[0, 0], [1, 2]] ] Copy to clipboard ### Inputs #### in[0] Input tensor - Mandatory: true - Data type: backend specific - Shape: tensor of rank N where dim[0] equals *batch* - Constraints: > > > - Shape: Rank > 0 #### in[1] indices : indices of batch to permute. Valid index values should be in range [0, batch - 1], otherwise they are ignored. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M], where M <= batch - Constraints: > > > - Value: Valid index values must be unique. ### Parameters None ### Outputs #### out[0] Output permuted tensor. Note that out[0] is ordered according to the indices provided that are in the range [0, batch - 1] and then padded with zeros. - Mandatory: true - Data type: backend specific - Shape: tensor of rank N where shape is the same as in[0] with the exception of dim[0] which equals *M*. ## BatchToSpace A type of tensor realignment operation that rearranges data from the batch dimension into blocks of spatial data, followed by cropping. The op moves blocks of data of size (block\_size[0] \* block\_size[1]) from the batch dimension of the input tensor into the spatial dimensions of the output tensor followed by cropping along the spatial dimensions. References: - TensorFlow: [ops::BatchToSpace](https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/batch-to-space) ### Inputs #### in[0] Input Activation. - Mandatory: true - Data type: backend specific - Shape: 4D tensor of shape [batch, height, width, depth] - Constraints: > > > - Shape: batch must be divisible by (block\_size[0] \* block\_size[1]) ### Parameters #### block\_size Vector that represents block size along the *height* and *width* dimensions respectively. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [block\_height, block\_width] - Constraints: > > > - Value: Elements must be >=1 #### crops Crop region that specifies how many elements to crop from the intermediate result across the spatial dimensions. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[crop\_top, crop\_bottom], [crop\_left, crop\_right]] - Default: [[0, 0], [0, 0]] ### Outputs #### out[0] Output Activation. Permuted output tensor with new spatial dimensions [*output\_height*, *output\_width*] defined by output_height = (height * block_size[0]) - crop_top - crop_bottom output_width = (width * block_size[1]) - crop_left - crop_right Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: 4D tensor of shape [batch / (block\_size[0] \* block\_size[1]), output\_height, output\_width, depth] - Constraints: > > > - Datatype: Same datatype as in[0] ## BboxTransform Transform bounding box proposals into refined bounding boxes using bounding box regression deltas for each class. Notes: - Bounding boxes can be rotated. - Resulting bounding boxes are clipped against the edges of the image. - The number of regions of interest (num\_rois) is in the range [0, N] where N is the maximum set by the underlying tensor. References: - QNN: [AxisAlignedBboxTransform](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#axisalignedbboxtransform) ### Inputs #### in[0] Bounding box proposal locations. - Mandatory: true - Data type: backend specific - Shape: [num\_rois, 5] each line with format [center\_x, center\_y, width, height, angle] #### in[1] Bounding box deltas for each region of interest and each class. Bounding box deltas are organized in the following format [dx, dy, dw, dh, da]. - Mandatory: true - Data type: backend specific - Shape: [num\_rois, num\_classes \* 5] #### in[2] Specifies image size. Image size is same for all images in a batch. - Mandatory: true - Data type: backend specific - Shape: [batches, 3] with format [image\_height, image\_width, image\_scale] #### in[3] Batch index of each bounding box. Batches with the same index are grouped together. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [num\_rois] - Default: {0,..,0} ### Parameters #### weights Weights applied to each of the bounding boxes deltas in in[1] in the form of (wx, wy, ww, wh). - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: [4] #### apply\_scale Set QNN\_OP\_BBOX\_TRANSFORM\_APPLY\_SCALE to true to transform the boxes to the scaled image space after applying the bounding box deltas, or QNN\_OP\_BBOX\_TRANSFORM\_APPLY\_SCALE to false not apply scale. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 #### angle\_bounds Limits the bounding box angle to be within the range [angle\_bound\_low, angle\_bound\_high]. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [2] with format [angle\_bound\_low, angle\_bound\_high] - Default: parameter not used unless set #### angle\_clip\_threshold Implements: angle = (angle < max(angle_clip_threshold, 0.0)) ? (0.0) : (angle) Copy to clipboard Set to negative to disable. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: -1.0 ### Outputs #### out[0] Coordinates of refined bounding boxes. - Mandatory: true - Data type: backend specific - Shape: [num\_rois, num\_classes \* 5] - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: same as in[1] #### out[1] Number of RoIs per batch. - Mandatory: true - Data type: backend specific - Shape: [batches] ## BoxWithNmsLimit Greedily selects a subset of bounding boxes in descending order of score. This op applies NMS algorithm to each class. In each loop of execution, the box with maximum score is selected and removed from the pending set. The scores of the rest of boxes are lowered according to the intersection-over-union (IOU) overlapping with the previously selected boxes and a specified NMS kernel method. Any boxes with score less than a threshold are removed from the pending set. Three NMS kernels are supported: Hard: score_new = score_old * (1 if IoU < threshold else 0) Linear: score_new = score_old * (1 if IoU < threshold else 1 - IoU) Gaussian: score_new = score_old * exp(- IoU^2 / sigma) Copy to clipboard Axis-aligned bounding boxes are represented by its upper-left corner coordinate (x1,y1) and lower-right corner coordinate (x2,y2). A valid bounding box should satisfy x1 <= x2 and y1 <= y2. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_BOX_WITH_NMS_LIMIT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a2d81e878c19e15700dad111ba6c0be89) ### Inputs #### in[0] Bounding box proposals. Elements can be understood as 4-tuples of bounding box coordinates given in the form (x1,y1,x2,y2). Boxes pertaining to a given batch element are grouped consecutively. - Mandatory: true - Data type: backend specific - Shape: [num\_rois, num\_classes\*4] #### in[1] Bounding box scores. The element at position [*roi*, *class*] can be understood as the score for the bounding box at the same position in in[0]. - Mandatory: true - Data type: backend specific - Shape: [num\_rois, num\_classes] #### in[2] Bounding box batch indices. Specifies the batch index of each box. Boxes pertaining to a given batch element are grouped consecutively. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [num\_rois] #### in[3] batch splits : Specifies the number of RoIs/boxes belonging to the corresponding image in batch. Note that the sum of values should add up to a total of num\_rois. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [batch] ### Parameters #### nms\_kernel\_method Determines the NMS kernel method, options are 0:hard, 1:linear, 2:gaussian. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Values: > > > - hard = 0, > - linear = 1, > - gaussian = 2 #### nms\_score\_threshold Boxes with scores lower than the threshold are dropped during the score updating phase in soft NMS. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar #### score\_threshold Boxes with scores lower than the threshold are filtered before sending to the NMS algorithm. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### pre\_nms\_limit Specifies a maximum number of boxes for each image which will be sent to NMS. Set to a negative value for unlimited number of output bounding boxes. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Default: -1 #### iou\_threshold Specifies the IoU threshold in hard and linear NMS kernel. This parameter is ignored if gaussian kernel is selected. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar #### sigma Specifies the sigma value in gaussian NMS kernel. This parameter is ignored if the gaussian kernel is not selected. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar ### Outputs #### out[0] Output boxes. Each element can be understood as a 4-tuple with the same meaning as in[0]. Boxes are grouped by batch, but order within each batch is not guaranteed. - Mandatory: true - Data type: backend specific - Shape: [num\_output\_rois, 4] - Constraints: > > > - Datatype: Same datatype as in[0] #### out[1] Output box scores. Gives the score for the box in the corresponding position in out[0]. - Mandatory: true - Data type: backend specific - Shape: [num\_output\_rois] - Constraints: > > > - Datatype: Same datatype as in[1] #### out[2] Output box classes. Gives the class index (with respect to *num\_classes* in in[0]) with the maximum score for the box in the corresponding position in out[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [num\_output\_rois] #### out[3] Output box batch indices : Gives the batch index for the box in the corresponding position in out[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [num\_output\_rois] #### out[4] Output batch splits : Specifies the number of RoIs/boxes belonging to the corresponding image in batch after applying NMS. Note that the sum of values should add up to a total of num\_output\_rois. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [batch] #### out[5] keeps : contains the indices of the selected boxes after performing NMS. The values of the indices are in the order of the boxes in out[0] and correspond to index position of the selected boxes in in[0]. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [num\_output\_rois] #### out[6] keeps size : contains the number of selected boxes per class after NMS has been applied. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [num\_classes] ## Buffer Accumulates input frames across inferences into a buffer of size *buffer\_size* and outputs the buffer with the collected frames. When the buffer is full the oldest existing frames in the buffer are removed to make space for the incoming new frame. The number of frames to remove is determined by *stride*. The remaining frames are shifted in the buffer to maintain the order they were received. ### Inputs #### in[0] input activation. Note that Shape(in[0])[buffer\_dim] determines the number of frames in the input. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] reset : Determines if the buffer should be reset. When set to true all frames in the buffer are removed. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: 0D containing scalar value - Default: 0 ### Parameters #### buffer\_size Determines the number of frames that a buffer can store. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be evenly divisible by Shape(in[0])[buffer\_dim]. #### buffer\_dim Determines the dimension that frames are accumulated on. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be in range [0, N-1] #### stride Determines the number of frames to remove from the buffer when the buffer is full to make space for the new incoming frames. The oldest existing frames which reside at the beginning of the buffer are removed. After removal the remaining frames are kept in the order they were received. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 - Constraints: > > > - Value: Must be in range [Shape(in[0])[buffer\_dim], buffer\_size]. > - Value: Must be evenly divisible by Shape(in[0])[buffer\_dim]. #### mode Determines blocking behavior. How the buffer is populated differs between the modes when the buffer is not full. When the buffer is full eviction and population behavior is the same for all modes. BLOCKING :0, NON\_BLOCKING\_LEFT: 1, NON\_BLOCKING\_RIGHT: 2. When mode is set to BLOCKING: Execution is stopped on the existing branch of the graph if the buffer is not full. The buffer is populated from the beginning to the end. For example, an empty buffer with 3 slots (0,1,2) will be populated from slot 0 to slot 2. When mode is set to NON\_BLOCKING\_LEFT: The existing branch of the graph will always execute regardless if the buffer is full or not. The buffer is populated from the beginning to the end. For example, an empty buffer with 3 slots (0,1,2) will be populated from slot 0 to slot 2. When mode is set to NON\_BLOCKING\_RIGHT: The existing branch of the graph will always execute regardless if the buffer is full or not. The buffer is populated from the end. For example, an empty buffer with 3 slots (0,1,2) the first incoming frame is placed at slot 2. For the next incoming frame, the previous frame at slot 2 is now at slot 1 and the new frame is placed at slot 2. When the buffer is full the number of frames removed is determined by *stride*. Eviction behavior is the same for all modes where the oldest existing frames are removed from the beginning of the buffer. Population behavior is also the same for all modes when the buffer is full. For example, a fully populated buffer with 3 slots (0,1,2) and a stride value of 2 will have the frames at slot 0 and slot 1 removed and the frame at slot 2 will now be at slot 0. The incoming frame is then placed at slot 1. The next incoming frame will then be placed at slot 2. When the buffer is full again the same process is repeated. Note that NON\_BLOCKING\_LEFT and NON\_BLOCKING\_RIGHT will be zero filled for the output if the buffer is not completely full. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - BLOCKING = 0, > - NON\_BLOCKING\_LEFT = 1, > - NON\_BLOCKING\_RIGHT = 2 #### buffer\_padding Determines the number of frames to pad with 0’s to the buffer initially or after reset. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: Same shape as in[0] except where dim[buffer\_dim] is equal to buffer\_size. - Constraints: > > > - Datatype: Same Datatype as in[0] > - Datatype: Same Rank as in[0] ## Cast Casts tensor data type to a new data type. This operation ignores quantization parameters specified with Qnn\_QuantizeParams\_t for tensors of fixed point data types, e.g. it treats a QNN\_DATATYPE\_UFIXED\_POINT\_8 tensor data type as a tensor of QNN\_DATATYPE\_UINT\_8 data type. Refer to Cast backend definition for support of dynamic dimensions for each backend. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_CAST](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a0a1bde5b34668d90eec4cfa4944e241d) - ONNX: [ops::Cast](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Cast) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: Any - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Same shape as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(out[0])[i] must be dynamic. ## ChannelShuffle This operation shuffles the channels of the input tensor, by dividing the *channel* dimension into *num\_groups* groups, and reorganizing the channels by grouping channels with the same index in each group. Along the *channel* dimension, the output is calculated using this formula: output_channel[i * num_groups + g] = input_channel[g * group_size + i] Copy to clipboard where num_channels = shape(in[0])[channel] group_size = num_channels / num_groups g is a group index : [0, num_groups-1] i is index within the group : [0, group_size-1] Copy to clipboard The *num\_channels* must be evenly divisible by *num\_groups*. *num\_groups* = *num\_channels* results in no-op. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_CHANNEL_SHUFFLE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a5b993c1211c4b1bc52fb595a3025251d) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters #### num\_groups Number of groups to divide *channel* dimension into, <= *num\_channels*. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar #### axis Axis on which channel shuffle will be performed - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: N-1 - Constraints: > > > - Value: Must be in range [0,N-1] ### Outputs #### out[0] output tensor with shuffled channels - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same rank as in[0] ## Col2Im Rearranges batched columns, each representing a snapshot over a kernel of an image, back into an image. The overlapping values in the resulting image are summed. References: - PyTorch: [Fold](https://pytorch.org/docs/stable/generated/torch.nn.Fold.html) - ONNX: [ops::Col2Im](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Col2Im) ### Inputs #### in[0] Input tensor of batched columns - Mandatory: true - Data type: backend specific - Shape: [batch, channel \* height\_kernel \* width\_kernel, L], where L is the number of columns - Constraints: > > > - Shape: The value of L must satisfy \(\prod\_i(\text{floor}(\frac{spatial\\_dimensions[i] + 2 \* padding[i]- dilation[i] \* (kernel\\_size[i] -1) - 1}{stride[i]} + 1))\), with spatial\_dimensions being the height and width of the output image. ### Parameters #### kernel\_size The size of the sliding block for retrieving the image. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_kernel, width\_kernel] #### stride Defines stride for 2D spatial (i.e. height and width) axes of the output image. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Paddings along the height and width dimensions of the output image. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[pad\_top, pad\_bottom], [pad\_left, pad\_right]] #### dilation Dilation value along each spatial axis (i.e. height and width) of the kernel for retrieving the image. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_dilation, width\_dilation] - Default: [1, 1] - Constraints: > > > - Value: Dilations must be > 0 ### Outputs #### out[0] Output tensor describing the shape of the resulting image - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel] - Constraints: > > > - Datatype: Same datatype as in[0] ## CollectRpnProposals Collect RoIs and their scores, merge predictions across multiple FPN levels, and retain the top scoring RoIs. Note that RoI elements can be understood as 5-tuples with format (batch\_idx, x1, y1, x2, y2) where batch\_idx specifies the batch index of each RoI, (x1,y1) represents the upper-left corner coordinate, and (x2,y2) represents the lower-right corner coordinate. ### Inputs #### in[0] RoIs : RPN proposals for FPN level *rpm\_min\_level*. - Mandatory: true - Data type: backend specific - Shape: [num\_rois, 5] each line with format [batch\_idx, x1, y1, x2, y2] #### in[1..4] RoIs : RPN proposals for FPN levels [*rpn\_min\_level* + 1, *rpn\_max\_level*]. Note that each input can have a different shape since *num\_roi\_i* can vary for in[1..4]. This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: false - Data type: backend specific - Shape: [num\_rois\_i, 5] each line with format [batch\_idx, x1, y1, x2, y2] - Constraints: > > > - Datatype: Same datatype as in[0] > - Number: Number of in[1..4] provided must equal rpn\_max\_level - rpn\_min\_level #### in[5] RoI probabilities : RPN scores for FPN level *rpn\_min\_level*. - Mandatory: true - Data type: backend specific - Shape: [num\_rois] #### in[6..9] RoI probabilities : RPN scores for FPN levels [*rpn\_min\_level* + 1, *rpn\_max\_level*]. Note that each input can have a different shape since *num\_roi\_i* can vary for in[6..9] but must be the same as the corresponding *RoIs* in[1..4] provided for the same FPN level. This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: false - Data type: backend specific - Shape: [num\_rois\_i] - Constraints: > > > - Datatype: Same datatype as in[5] > - Shape: shape(in[i])[0] must equal shape(in[i+5])[0] for i=1..4 > - Number: Number of in[6..9] provided must equal rpn\_max\_level - rpn\_min\_level ### Parameters #### rpn\_min\_level Sets the minimum FPN level to support RPN transform operations on multiple FPN levels - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 2 - Constraints: > > > - Value: Must be in range [2,6] #### rpn\_max\_level Sets the maximum FPN level to support RPN transform operations on multiple FPN levels. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 6 - Constraints: > > > - Value: Must be in range [2,6] > - Value: Must be >= rpn\_min\_level #### post\_nms\_top Sets a maximum number of proposals. The proposals with the lowest scores will be dropped to achieve this limit. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 2000 ### Outputs #### out[0] RoIs : Top proposals limited to *post\_nms\_top* total. - Mandatory: true - Data type: backend specific - Shape: [num\_proposals, 5] each line with format [batch\_idx, x1, y1, x2, y2], where total\_num\_rois is the sum of all *num\_rois* from in[0..4] and num\_proposals = min(*total\_num\_rois*, *post\_nms\_top*). ## CombinedNms Performs Non Max Suppression (NMS) on the inputs per batch, across all classes. Boxes that have a high intersection-over-union (IOU) overlap with previously selected boxes are pruned. Additionally, bounding boxes with scores less than *score\_threshold* are pruned. NMS can be either class-specific or class-agnostic depending on the shape of the in[0] that is provided. For the class-specific case NMS is performed independently for each class and class-agnostic case NMS will ignore all class labels and compare all bounding boxes. References: - TensorFlow: [CombinedNonMaxSuppression](https://www.tensorflow.org/api_docs/python/tf/image/combined_non_max_suppression) ### Inputs #### in[0] Bounding boxes: Elements can be understood as 4-tuples of bounding box coordinates given in the form (y1,x1,y2,x2), where (y1, x1) and (y2, x2) represent a diagonal pair of corners. Coordinates can be provided as normalized or absolute. Note that if q is equal to *num\_classes* then class-specific boxes are used otherwise if q is equal to 1 then same boxes are used for all classes. - Mandatory: true - Data type: backend specific - Shape: [batch, num\_boxes, q, 4] - Constraints: > > > - Shape: q must be equal to 1 or num\_classes #### in[1] Bounding box scores. The element at position [*batch*, *box*, *class*] is the score corresponding to *class* for the bounding box at the position [*batch*, *box*, *q*] in in[0]. - Mandatory: true - Data type: backend specific - Shape: [batch, num\_boxes, num\_classes] ### Parameters #### max\_boxes\_per\_class Maximum number of boxes that can be selected by NMS per class. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar #### max\_total\_boxes Maximum number of boxes to be retained over all classes after applying NMS. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar #### iou\_threshold Represents the threshold used by NMS algorithm to determine whether boxes overlap too much. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Constraints: > > > - Value: Must be in range [0, 1] #### score\_threshold Boxes with scores lower than the threshold are filtered out by the NMS algorithm. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### pad\_per\_class If true, the output boxes, scores, and classes padded to the size of min(max\_boxes\_per\_class \* num\_classes, max\_total\_boxes). Otherwise, the output boxes, scores, and classes are padded to the size of max\_total\_boxes. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### clip\_boxes If true, the coordinates of selected output boxes in out[0] are clipped to [0, 1]. Otherwise, no clipping is done. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Selected Output boxes. Each element can be understood as a 4-tuple with the same meaning as in[0]. Note that the number of valid boxes is specified by out[3]. - Mandatory: true - Data type: backend specific - Shape: [batch, max\_num\_boxes, 4], where max\_num\_boxes = min(max\_boxes\_per\_class \* num\_classes, max\_total\_boxes) if pad\_per\_class is true. Otherwise, max\_num\_boxes = max\_total\_boxes. - Constraints: > > > - Datatype: Same datatype as in[0] #### out[1] Selected Output box scores. Gives the score for the box in the corresponding position in out[0]. Note that the number of valid scores is specified by out[3]. - Mandatory: true - Data type: backend specific - Shape: [batch, max\_num\_boxes], where max\_num\_boxes = min(max\_boxes\_per\_class \* num\_classes, max\_total\_boxes) if pad\_per\_class is true. Otherwise, max\_num\_boxes = max\_total\_boxes. - Constraints: > > > - Datatype: Same datatype as in[1] #### out[2] Selected Output classes. Gives the class label for the box in the corresponding position in out[0]. Note that the number of valid classes is specified by out[3]. - Mandatory: true - Data type: backend specific - Shape: [batch, max\_num\_boxes], where max\_num\_boxes = min(max\_boxes\_per\_class \* num\_classes, max\_total\_boxes) if pad\_per\_class is true. Otherwise, max\_num\_boxes = max\_total\_boxes. #### out[3] Number of valid boxes per batch element that remain after NMS. Only the top entries of the output boxes, scores, and classes are valid. All other entries are zero padding. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [batch] - Constraints: > > > - Value: <= max\_num\_boxes ## Concat Concatenates two or more input tensors along a provided axis. Refer to Concat backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_CONCATENATION](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a44cbea825c4b224dd3ea757e9b1f65ed) ### Inputs #### in[0..m] input tensors. m >=1 This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Number: m >= 1 > - Shape: Rank > 0 > - Shape: shape(in[0..m])[i] must be the same shape for all inputs, except at shape(in[0..m])[*axis*], which is permitted to vary across inputs. > - Dynamic Shape: For any input, if shape(in[0..m])[i] is dynamic then shape(in[0..m])[i] must be dynamic across all inputs, except at the *axis* dimension. ### Parameters #### axis Axis on which to concatenate input tensors - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: N-1 - Constraints: > > > - Value: Must be in range [0,N-1] ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Same rank as in[0] > - Shape: Same shape as in[0], except shape(out[0])[axis] = sum(i, shape(in[i])[axis]) > - Dynamic Shape: If a dynamic dimension exists at shape(in[0..m])[*axis*] for any input then shape(out[0])[*axis*] must be dynamic. For all other dimensions, if shape(in[0..m])[i] is dynamic across all inputs then shape(out[0])[i] must be dynamic. ## ConstantOfShape Generates an output tensor with the given shape and value. References: - ONNX: [ops::ConstantOfShape](https://github.com/onnx/onnx/blob/main/docs/Operators.md#ConstantOfShape) - PyTorch: [Full](https://pytorch.org/docs/stable/generated/torch.full.html) - TensorFlow: [Fill](https://www.tensorflow.org/api_docs/python/tf/fill) ### Inputs #### in[0] Input tensor : a 1D tensor specifying the shape of the expected output tensor. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [N] ### Parameters #### value The value to fill the output tensor with. - Mandatory: true - Data type: backend specific - Shape: scalar ### Outputs #### out[0] Output tensor with shape specified by in[0] and value specified by *value*. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as *value*. ## Conv1d Performs 1D convolution: dot-product of a set of 1D filters with input activation, producing output activation. Application of the filters moves according to the specified stride. For backends supporting quantized data types, clients can pass filters which are either quantized per-tensor or per-axis with possible constraints on the axis value that is supported. For regular convolution, *group* is 1. Group field greater than 1 implies a grouped convolution where a group of different filters is applied to each input channel group and the result is concatenated together. References: - TensorFlow: [Conv1D](https://www.tensorflow.org/api_docs/python/tf/nn/conv1d) - Onnx: [Conv](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Conv) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, width, channel\_in] - Constraints: > > > - Shape: channel\_in must be evenly divisible by group #### in[1] filters - Mandatory: true - Data type: backend specific - Shape: [filter\_width, channel\_in / group, channel\_out] - Constraints: > > > - Shape: channel\_out must be evenly divisible by group #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [channel\_out] - Default: {0,..,0} ### Parameters #### stride Defines stride for 1D spatial axes of in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 1D spatial axes of in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [width\_pad\_before, width\_pad\_after] #### group - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 #### dilation Dilation parameter for width dimension. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 - Constraints: > > > - Value: Must be > 0 ### Outputs #### out[0] The output 1D spatial dimension is a function of the filter size, stride, and pad\_amount. dilated_filter_width = (shape(in[1])[width] - 1) * dilation + 1 width_out = floor((pad_amount[0] + shape(in[0])[width] + pad_amount[1] - dilated_filter_width) / stride + 1) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, width\_out, channel\_out] ## Conv2d Performs 2D convolution: dot-product of a set of 2D filters with input activation, producing output activation. Application of the filter moves according to the specified strides. For backends supporting quantized data types, clients can pass filters which are either quantized per-tensor or per-axis with possible constraints on the axis value that is supported. For regular convolution, *group* is 1. Group field greater than 1 implies a grouped convolution where a group of different filters is applied to each input channel group and the result is concatenated together. Note that *channel\_out* and *channel\_in* must be evenly divisible by *group*. Refer to Conv2d backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_CONV_2D](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a34a73b5eaf458b67db5eda71557d1d01) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel\_in] - Dynamic Shape: All dimensions can be dynamic. #### in[1] filters - Mandatory: true - Data type: backend specific - Shape: [filter\_height, filter\_width, channel\_in / group, channel\_out] - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: shape(in[0])[channel\_in] and shape(in[1])[channel\_in/group] must both be dynamic or both static. #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [channel\_out] - Dynamic Shape: All dimensions can be dynamic. - Default: {0,..,0} - Constraints: > > > - Dynamic Shape: shape(in[1])[channel\_out] and shape(in[2])[channel\_out] must both be dynamic or both static. ### Parameters #### stride Defines stride for 2D spatial axes of in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 2D spatial axes of in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### group - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 #### dilation Dilation parameter for height and width dimensions. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [height\_dilation, width\_dilation] - Default: [1, 1] - Constraints: > > > - Value: Dilations must be > 0 #### reuse\_sparse\_indices Only for sparse input and output tensors. If true, the resulting convolution re-uses the input indices for the output indices. Convolutions are only computed for the specified elements. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] The output 2D spatial dimensions are functions of the filter size, stride, and pad\_amount. dilated_filter_height = (shape(in[1])[filter_height] - 1) * dilation[0] + 1 dilated_filter_width = (shape(in[1])[filter_width] - 1) * dilation[1] + 1 height_out = floor((pad_amount[0,0] + shape(in[0])[height] + pad_amount[0,1] - dilated_filter_height) / stride[0] + 1) width_out = floor((pad_amount[1,0] + shape(in[0])[width] + pad_amount[1,1] - dilated_filter_width) / stride[1] + 1) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, height\_out, width\_out, channel\_out] - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: If shape(in[0])[batch] is dynamic, shape(out[0])[batch] must also be dynamic. If shape(in[0])[height] or shape(in[1])[filter\_height] are dynamic, then shape(out[0])[height\_out] must be dynamic. If shape(in[0])[width] or shape(in[1])[filter\_width] are dynamic, then shape(out[0])[width\_out] must be dynamic. If *in[2]* is not provided and shape(in[1])[channel\_out] is dynamic or if *in[2]* is provided and both shape(in[1])[channel\_out] and shape(in[2])[channel\_out] are dynamic, then shape(out[0])[channel\_out] must also be dynamic. > - Must have same data format as in[0] (e.g. both sparse or both dense) ## Conv3d Performs 3D convolution: a spatial convolution over volumes using a set of 3D filters with input activation, producing output activation. Application of the filter moves according to the specified strides. For backends supporting quantized data types, clients can pass filters which are either quantized per-tensor or per-axis with possible constraints on the axis value that is supported. For regular convolution, *group* is 1. Group field greater than 1 implies a grouped convolution where a group of different filters is applied to each input channel group and the result is concatenated together. Note that *channel\_out* and *channel\_in* must be evenly divisible by *group*. References: - TensorFlow: [Conv3D](https://www.tensorflow.org/api_docs/python/tf/nn/conv3d) - Onnx: [Conv](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Conv) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, depth, height, width, channel\_in] #### in[1] filters - Mandatory: true - Data type: backend specific - Shape: [filter\_depth, filter\_height, filter\_width, channel\_in / group, channel\_out] #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [channel\_out] - Default: {0,..,0} ### Parameters #### stride Defines stride for 3D spatial axes of in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [depth\_stride, height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 3D spatial axes of in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3,2] with format [[depth\_pad\_before, depth\_pad\_after], [height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### group - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 #### dilation Dilation parameter for depth, height and width dimensions. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [depth\_dilation, height\_dilation, width\_dilation] - Default: [1, 1, 1] - Constraints: > > > - Value: Dilations must be > 0 #### reuse\_sparse\_indicies Only for sparse input and output tensors. If true, the resulting convolution re-uses the input indices for the output indices. Convolutions are only computed for the specified elements. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] The output 3D spatial dimensions depth, height, and width are functions of the *filters* size, *stride*, and *pad\_amount*. dilated_filter_depth = (shape(in[1])[depth] - 1) * dilation[0] + 1 dilated_filter_height = (shape(in[1])[height] - 1) * dilation[1] + 1 dilated_filter_width = (shape(in[1])[width] - 1) * dilation[2] + 1 depth_out = floor((pad_amount[0,0] + shape(in[0])[depth] + pad_amount[0,1] - dilated_filter_depth) / stride[0] + 1) height_out = floor((pad_amount[1,0] + shape(in[0])[height] + pad_amount[1,1] - dilated_filter_height) / stride[1] + 1) width_out = floor((pad_amount[2,0] + shape(in[0])[width] + pad_amount[2,1] - dilated_filter_width) / stride[2] + 1) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, depth\_out, height\_out, width\_out, channel\_out] - Constraints: > > > - Datatype: Same datatype as in[0] > - Must have same data format as in[0] (e.g. both sparse or both dense) ## Convert This operation converts input activation tensor to output activation tensor as per corresponding tensor data type. Unlike in [Cast](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#cast) operation, quantization parameters as specified with Qnn\_QuantizeParams\_t are obeyed for fixed point data type conversions. The operation also provides optional support for data type changes at runtime. Refer to Convert backend definition for support of dynamic dimensions for each backend. Backends do not support dynamic dimensions unless stated otherwise. References: - QNN: [Cast](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#cast) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: Any - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### dynamic\_input\_data Set QNN\_OP\_CONVERT\_PARAM\_DYNAMIC\_INPUT\_DATA to *true* to indicate that in[0] data type and associated buffer can change in between op execute invocations. It means that client is allowed to change data type and associated buffer of adequate size before QnnGraph\_execute() call, subject to constraints and backend support. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 - Constraints: > > > - Value: *true* is valid/allowed only for tensors of type QNN\_TENSOR\_TYPE\_APP\_WRITE or QNN\_TENSOR\_TYPE\_APP\_READWRITE. #### dynamic\_output\_data Set QNN\_OP\_CONVERT\_PARAM\_DYNAMIC\_OUTPUT\_DATA to *true* to indicate that out[0] data type and associated buffer can change in between op execute invocations. It means that client is allowed to change data type and associated buffer of adequate size before QnnGraph\_execute() call, subject to constraints and backend support. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 - Constraints: > > > - Value: *true* is valid/allowed only for tensors of type QNN\_TENSOR\_TYPE\_APP\_READ or QNN\_TENSOR\_TYPE\_APP\_READWRITE. ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Same shape as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(out[0])[i] must be dynamic. ## Correlation1D Performs a depth-wise one-dimensional correlation along the width axis as shown: \[n \in [0,\mbox{batch})\] \[h \in [0,\mbox{height})\] \[w \in [0,\mbox{width})\] \[d \in [0,\mbox{depth})\] \[w' = w + d - (\mbox{displacement} - \mbox{shift})\] \[\mbox{When } w' < 0 \mbox{ or } w' \ge \mbox{ width}: \mbox{out[0]}[n,h,w,d] = 0\] \[\mbox{Otherwise}: \mbox{out[0]}[n,h,w,d] = \frac{\sum^{\mbox{depth}-1}\_{d'=0}\mbox{in[0]}[n,h,w,d']\cdot\mbox{in[1]}[n,h,w',d']}{\mbox{depth}}\] References: - FlowNet 2.0: [Correlation 1D](https://github.com/lmb-freiburg/flownet2/blob/master/src/caffe/layers/correlation_layer1d.cpp) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, depth] #### in[1] - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, depth] - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters #### displacement Maximum searching pixels in the horizontal direction - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar #### shift - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, displacement\*2 + 1] - Constraints: > > > - Datatype: Same datatype as in[0] ## CreateSparse Creates a sparse tensor from indices and values tensors. Unspecified elements are considered to have a zero value. For quantized data types, this also implies that the offset is zero. Note that sparse tensors can be partially sparse. For example, we accommodate cases where spatial dimensions are sparse, but the channel dimension is dense (e.g. RGB). We manage this with the definition of K, which represents the number of sparse dimensions. Note that all K sparse dimensions must be the outermost (slowest changing) dimensions. The N-K dense dimensions are the innermost (fastest changing) dimensions. References: - PyTorch: [torch.sparse](https://pytorch.org/docs/stable/sparse.html) - TensorFlow: [tf.sparse](https://www.tensorflow.org/api_docs/python/tf/sparse) - QNN: [GetSparseIndices](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#getsparseindices) - QNN: [GetSparseValues](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#getsparsevalues) - QNN: [SparseToDense](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#sparsetodense) ### Inputs #### in[0] indices The elements of the in[0] indices tensor correspond to the respective element of the in[1] values tensor in the equivalent dense tensor. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32, QNN\_DATATYPE\_INT\_32 - Shape: [M, K] where 0 < K <= N. M may be dynamically sized and K is fixed. #### in[1] values - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N-K+1 with shape \([M,D\_{K},...,D\_{N-1}]\) ### Parameters None ### Outputs #### out[0] output - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N with shape \([D\_0,...,D\_{N-1}]\) - Constraints: > > > - Datatype: Same datatype as in[1] > - Must be a sparse tensor. ## CropAndResize Extract crops from the input batch of images and resize them to a common specified output size. This operation may not preserve aspect ratio of the input crops. The resizing is corner-aligned. References: - TensorFlow: [crop_and_resize](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/image/crop_and_resize) ### Inputs #### in[0] Input activation - Mandatory: true - Data type: backend specific - Shape: [batch,height,width,channel] #### in[1] Crop boxes : Each box defines the crop zone in one of the images in the input batch. Elements may be interpreted as 4-tuples of (y1,x1,y2,x2) representing normalized crop coordinates. A normalized coordinate value of y is mapped to the image coordinate at y \* (image\_height - 1), so as the [0, 1] interval of normalized image height is mapped to [0, image\_height - 1] in image height coordinates. The condition (y1 > y2) is permitted, in which case the sampled crop is an up-down flipped version of the original image. The width dimension is treated similarly. Specifying coordinates outside of the range [0, 1] result in extrapolation of input image values. - Mandatory: true - Data type: backend specific - Shape: [num\_boxes,4] #### in[2] Batch index in the input tensor to which each box corresponds. Indices in in[2] indicate the images in the input batch that will be cropped using box coordinates in the same position in in[1]. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32 - Shape: [num\_boxes] - Constraints: > > > - Value: in range [0, batch-1] ### Parameters #### resize\_dims The dimensions to which input images are cropped and resized to [resize\_height, resize\_width]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] #### interpolation\_mode Determines the interpolation method. Supported values are 0: BILINEAR, 1: NEAREST\_NEIGHBOR. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - BILINEAR = 0, > - NEAREST\_NEIGHBOR = 1 #### extrapolation\_value Value used for extrapolation during the resize operation when applicable. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output activation - Mandatory: true - Data type: backend specific - Shape: [num\_boxes,resize\_height,resize\_width,channel]. - Constraints: > > > - Datatype: Same datatype as in[0] ## CumulativeSum Performs cumulative sum of the input elements along the given axis. By default, this op performs the sum inclusively meaning the first element of the input is identical to the first element of the output. An exclusive sum can be performed by setting the *exclusive* parameter to true. It can also perform summation in the opposite direction of the axis by setting the *reverse* parameter to true. // Default case When QNN_OP_CUMULATIVE_SUM_PARAM_EXCLUSIVE = 0 AND QNN_OP_CUMULATIVE_SUM_PARAM_REVERSE = 0 : output[0] = input[0] is always true (for i = 0) output[i] = input[0] + input[1] + ... + input[i] (for i > 0) // Example input[axis] = [1, 2, 3], exclusive = 0, reverse = 0 output[axis] = [1, 3, 6] input[axis] = [1, 2, 3], exclusive = 1, reverse = 0 output[axis] = [0, 1, 3] input[axis] = [1, 2, 3], exclusive = 0, reverse = 1 output[axis] = [6, 5, 3] input[axis] = [1, 2, 3], exclusive = 1, reverse = 1 output[axis] = [5, 3, 0] Copy to clipboard References: - ONNX: [CumSum](https://github.com/onnx/onnx/blob/master/docs/Operators.md#CumSum) - TensorFlow: [ops::Cumsum](https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/cumsum) ### Inputs #### in[0] Input data - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axis Dimension index, starts at 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be in range [0, rank(in[0]) - 1] #### exclusive If QNN\_OP\_CUMULATIVE\_SUM\_PARAM\_EXCLUSIVE is set to true an exclusive sum is performed where the i-th output element is the sum of the first (i - 1) elements. When QNN_OP_CUMULATIVE_SUM_PARAM_EXCLUSIVE = 1 AND QNN_OP_CUMULATIVE_SUM_PARAM_REVERSE = 0 : output[0] = 0 is always true (for i = 0) output[i] = input[0] + input[1] + ... + input[i - 1] (for i > 0) Copy to clipboard - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### reverse If QNN\_OP\_CUMULATIVE\_SUM\_PARAM\_REVERSE is set to true the sum is performed in the reverse direction. When QNN_OP_CUMULATIVE_SUM_PARAM_EXCLUSIVE = 0 AND QNN_OP_CUMULATIVE_SUM_PARAM_REVERSE = 1 : output[K] = input[K] is always true (for i = K, where K = input.dim[axis] - 1) output[i] = input[K] + input[K - 1] + ... + input[i + 1] + input[i] (for 0 <= i < K) When QNN_OP_CUMULATIVE_SUM_PARAM_EXCLUSIVE = 1 AND QNN_OP_CUMULATIVE_SUM_PARAM_REVERSE = 1 : output[K] = 0 is always true (for i = K, where K = input.dim[axis] - 1) output[i] = input[K] + input[K - 1] + ... + input[i + 2] input[i + 1] (for 0 <= i < K) Copy to clipboard - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output data - Mandatory: true - Data type: backend specific - Shape: Same shape as in[0] - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## DepthToSpace A type of tensor realignment operation that rearranges depth data into blocks of spatial data. The op moves blocks of data of size (block\_size[0] \* block\_size[1]) from the depth dimension of the input tensor into the spatial dimensions of the output tensor. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_DEPTH_TO_SPACE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a34253f8b844b4c143f0fa36be3ba3f7a) ### Inputs #### in[0] Input Activation. - Mandatory: true - Data type: backend specific - Shape: 4D tensor of shape [batch, height, width, depth] - Constraints: > > > - Shape: depth must be divisible by (block\_size[0] \* block\_size[1]) ### Parameters #### block\_size Vector that represents block size along the *height* and *width* dimensions respectively. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [block\_height, block\_width] - Constraints: > > > - Value: Elements must be >=1 #### mode Specifies the order in which elements of in[0] are rearranged. If QNN\_OP\_DEPTH\_TO\_SPACE\_PARAM\_MODE is set to QNN\_OP\_DEPTH\_TO\_SPACE\_MODE\_DCR then elements along the depth dimension are rearranged in the order of depth, column, and then row; if set to QNN\_OP\_DEPTH\_TO\_SPACE\_MODE\_CRD elements along the depth dimension are rearranged in the order of column, row, and then depth. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - DCR = 0, > - CRD = 1 ### Outputs #### out[0] Output Activation. - Mandatory: true - Data type: backend specific - Shape: 4D tensor of shape [batch, (height \* block\_size[0]), (width \* block\_size[1]), depth / (block\_size[0] \* block\_size[1])] - Constraints: > > > - Datatype: Same datatype as in[0] ## DepthWiseConv1d Performs depthwise 1D convolution: dot-product of a set of 1D filters with input activation, producing output activation. Depthwise 1D convolution applies a different filter to each input channel group, then concatenates the results together. Depthwise 1D convolution is functionally equivalent to Conv 1D where ‘group’ parameter value == channel\_in and channel\_out is a multiple of channel\_in, e.g. channel\_out % channel\_in == 0. Application of the filter moves according to the specified stride. For backends supporting quantized data types, clients can pass filters which are either quantized per-tensor or per-axis with possible constraints on the axis value that is supported. ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, width, channel\_in] #### in[1] filters - Mandatory: true - Data type: backend specific - Shape: [filter\_width, 1, channel\_out] - Constraints: > > > - Shape: Channel\_out must be divisible by channel\_in #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [channel\_out] - Default: {0,..,0} ### Parameters #### stride Defines stride for 1D spatial axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 1D spatial axes in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [width\_pad\_before, width\_pad\_after] #### dilation Dilation parameter for width dimension of in[0]. If set to d > 1, there will be d-1 skipped cells between each filter element on corresponding dimension. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 - Constraints: > > > - Value: Must be > 0 ### Outputs #### out[0] output activation : The output 1D spatial dimensions are functions of the *filter\_size*, *stride*, and *pad\_amount*. dilated_filter_width = (shape(in[1])[width] - 1) * dilation + 1 width_out = floor((pad_amount[0] + shape(in[0])[width] + pad_amount[1] - dilated_filter_width) / stride + 1) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, width\_out, channel\_out] - Constraints: > > > - Datatype: Same datatype as in[0] ## DepthWiseConv2d Performs depthwise 2D convolution: dot-product of a set of 2D filters with input activation, producing output activation. Depthwise 2D convolution applies a different filter to each input channel group, then concatenates the results together. Depthwise 2D convolution is functionally equivalent to Conv 2D where ‘group’ parameter value == channel\_in and channel\_out is a multiple of channel\_in, e.g. channel\_out % channel\_in == 0. Application of the filter moves according to the specified strides. For backends supporting quantized data types, clients can pass filters which are either quantized per-tensor or per-axis with possible constraints on the axis value that is supported. Refer to DepthWiseConv2d backend definition for supported data type and layouts for each backend. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_DEPTHWISE_CONV_2D](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a2b49a44b7ebba243fad01556c1f0392e) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel\_in] - Dynamic Shape: All dimensions can be dynamic. #### in[1] filters - Mandatory: true - Data type: backend specific - Shape: [filter\_height, filter\_width, 1, channel\_out] - Dynamic Shape: All dimensions can be dynamic with the exception of shape(in[1])[2]. #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [channel\_out] - Dynamic Shape: All dimensions can be dynamic. - Default: {0,..,0} - Constraints: > > > - Dynamic Shape: shape(in[1])[channel\_out] and shape(in[2])[channel\_out] must both be dynamic or both static. ### Parameters #### stride Defines stride for 2D spatial axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_stride, width\_stride] #### pad\_amount Pad amount to be added to the beginning and end part of 2D spatial axes in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### dilation Dilation parameter for height and width dimensions. If set to d > 1, there will be d-1 skipped cells between each filter element on corresponding dimension. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [height\_dilation, width\_dilation] - Default: [1, 1] - Constraints: > > > - Value: Dilations must be > 0 ### Outputs #### out[0] output activation The output 2D spatial dimensions are functions of the filter size, stride, and pad\_amount. dilated_filter_height = (shape(in[1])[height] - 1) * dilation[0] + 1 dilated_filter_width = (shape(in[1])[width] - 1) * dilation[1] + 1 height_out = floor((pad_amount[0,0] + shape(in[0])[height] + pad_amount[0,1] - dilated_filter_height) / stride[0] + 1) width_out = floor((pad_amount[1,0] + shape(in[0])[width] + pad_amount[1,1] - dilated_filter_width) / stride[1] + 1) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, height\_out, width\_out, channel\_out] - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: If shape(in[0])[batch] is dynamic, shape(out[0])[batch] must also be dynamic. If shape(in[0])[height] or shape(in[1])[filter\_height] are dynamic, then shape(out[0])[height\_out] must be dynamic. If shape(in[0])[width] or shape(in[1])[filter\_width] are dynamic, then shape(out[0])[width\_out] must be dynamic. If *in[2]* is not provided and shape(in[1])[channel\_out] is dynamic or if *in[2]* is provided and both shape(in[1])[channel\_out] and shape(in[2])[channel\_out] are dynamic, then shape(out[0])[channel\_out] must also be dynamic. ## Dequantize Dequantizes the input tensor. Note that *scale* and *offset* are determined from in[0]. Implements: output = (input - offset) * scale. Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_DEQUANTIZE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ad4c9300b061d9d14669bd5acdc7538e2) - QNN: [Quantize](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#quantize) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_SFIXED\_POINT\_4, QNN\_DATATYPE\_UFIXED\_POINT\_4, QNN\_DATATYPE\_SFIXED\_POINT\_8, QNN\_DATATYPE\_UFIXED\_POINT\_8, QNN\_DATATYPE\_SFIXED\_POINT\_16, QNN\_DATATYPE\_UFIXED\_POINT\_16, QNN\_DATATYPE\_SFIXED\_POINT\_32, QNN\_DATATYPE\_UFIXED\_POINT\_32, backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, backend specific - Shape: - Constraints: > > > - Shape: Same shape as in[0] ## DetectionOutput Decodes a set of bounding boxes from a set of pre-defined anchors, then filters boxes using non-max-suppression (NMS). References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_DETECTION_POSTPROCESSING](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0abd6365933837275bb1f5cde1fd9b8234) ### Inputs #### in[0] Scores for each class at each pre-defined anchor. - Mandatory: true - Data type: backend specific - Shape: [batch, num\_anchors, num\_classes] #### in[1] Input box locations. Elements may be interpreted as [ctr\_y, ctr\_x, h, w] where ctr\_y and ctr\_x give the center position of the box, and h and w are the height and width of the box. The number of input boxes is computed as follows: num_boxes = num_anchors if share_location = QNN_OP_DETECTION_OUTPUT_SHARE_LOCATION true num_boxes = (num_anchors * num_classes) if share_location = QNN_OP_DETECTION_OUTPUT_SHARE_LOCATION false Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, num\_boxes, 4] #### in[2] Anchor positions. Elements may be interpreted as [ctr\_y, ctr\_x, h, w] where ctr\_y and ctr\_x are the center position, and h and w are the height and width of the anchor. - Mandatory: true - Data type: backend specific - Shape: [num\_anchors \* batch, 4] ### Parameters #### delta\_scaling\_factors Multiplicative scaling factors applied to each of the bounding boxes in in[1] in the form of (dy, dx, dh, dw), where dy and dx are linear-scale shifts with respect to width and height, and dh and dw are log-scale scaling factors with respect to the width and height of the boxes. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: [4] #### confidence\_threshold Boxes with scores lower than this threshold are filtered prior to the application of NMS. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar #### iou\_threshold IoU threshold for the NMS algorithm. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar #### nms\_type Specifies which variant of NMS to use. Set QNN\_OP\_DETECTION\_OUTPUT\_NMS\_TYPE\_REGULAR for regular multi-class NMS, or QNN\_OP\_DETECTION\_OUTPUT\_NMS\_TYPE\_FAST for a faster variant which limits the number of classes to which NMS is applied. REGULAR: 1, FAST: 0 - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - FAST = 0, > - REGULAR = 1 #### background\_class\_idx The index in num\_classes of the “background” class. This class will be ignored by NMS. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 #### use\_bg\_in\_nms Choose to include background class in computing NMS. Set QNN\_OP\_DETECTION\_OUTPUT\_USE\_BG\_IN\_NMS true to include the BG class, or to false to ignore. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### output\_background Set to QNN\_OP\_DETECTION\_OUTPUT\_OUTPUT\_BACKGROUND true to include the background class in the output, or false to exclude the class. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 #### share\_location Set to QNN\_OP\_DETECTION\_OUTPUT\_SHARE\_LOCATION true to indicate that the classes all share a common set of initial bounding boxes, and QNN\_OP\_DETECTION\_OUTPUT\_SHARE\_LOCATION false to indicate that they use different initial bounding boxes. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 #### nms\_eta Adaptation factor for the NMS threshold. This factor is applied when nms\_type is set to QNN\_OP\_DETECTION\_OUTPUT\_NMS\_TYPE\_REGULAR. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1.0 #### detection\_limit Parameter that specifies: (i) the maximum number of classes per detection when nms\_type is set to QNN\_OP\_DETECTION\_OUTPUT\_NMS\_TYPE\_FAST. (ii) the maximum number of detections when applying NMS for each single class when nms\_type is set to QNN\_OP\_DETECTION\_OUTPUT\_NMS\_TYPE\_REGULAR. Parameter is ignored if set to default value. This parameter is similar to ‘nms\_topK’ found in training frameworks like Caffe which set nms\_type to QNN\_OP\_DETECTION\_OUTPUT\_NMS\_TYPE\_REGULAR. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Default: -1 ### Outputs #### out[0] Detection scores. - Mandatory: true - Data type: backend specific - Shape: [batch, max\_num\_detections]. max\_num\_detections can be expressed by maxDimensions in Qnn\_Tensor\_t. - Constraints: > > > - Datatype: Same datatype as in[0] #### out[1] Detection locations. Elements specify the coordinates of the output bounding boxes and can interpreted as 4-tuples with format (y1,x1,y2,x2) representing the upper-left and lower-right corner coordinates. A valid bounding box should satisfy x1 <= x2 and y2 <= y1. Note that quantized data types may provide output box coordinates that are out of range. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: [batch, max\_num\_detections, 4]. max\_num\_detections can be expressed by maxDimensions in Qnn\_Tensor\_t. #### out[2] Detection labels. Gives the class label for each detection. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [batch, max\_num\_detections] #### out[3] Valid number of detections per batch. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [batch] ## DistributeFpnProposals Distribute RPN proposals to their appropriate FPN levels for Faster RCNN. Note that RoI elements can be understood as 5-tuples with format (batch\_idx, x1, y1, x2, y2) where batch\_idx specifies the batch index of each RoI, (x1,y1) represents the upper-left corner coordinate, and (x2,y2) represents the lower-right corner coordinate. ### Inputs #### in[0] RoIs : RPN proposals. - Mandatory: true - Data type: backend specific - Shape: [num\_proposals, 5] each line with format [batch\_idx, x1, y1, x2, y2] or [num\_proposals, 4] each line with format [x1, y1, x2, y2] ### Parameters #### roi\_min\_level Sets the maximum FPN level to support RoI transform operations on multiple FPN levels. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 2 - Constraints: > > > - Value: Must be in range [2,5] #### roi\_max\_level Sets the maximum FPN level to support RoI transform operations on multiple FPN levels. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 5 - Constraints: > > > - Value: Must be in range [2,5] > - Value: Must be >= roi\_min\_level #### roi\_canonical\_scale Scaling factor used to compute which FPN level each RoI in a set of RoIs should map to. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 224 #### roi\_canonical\_level Value to offset the computed FPN level for each RoI. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 4 ### Outputs #### out[0] rois\_idx\_restore : indices to restore the RoIs to their original order from in[0]. Indices are in respect to the concatenation of the *RoIsFPN* outputs in order from FPN level *roi\_min\_level* to FPN level *roi\_max\_level*. For invalid RoIs having coordinate values (x2 - x1 = 0) and (y2 - y1 = 0) the corresponding index value will be set to -1. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32 - Shape: [num\_proposals] - Constraints: > > > - Value: Must be in range [0, num\_proposals \* (roi\_max\_level \* roi\_min\_level + 1) - 1] #### out[1] RoIs FPN : RoIs mapped to FPN level *roi\_min\_level*. - Mandatory: true - Data type: backend specific - Shape: [num\_rois, 5] each line with format [batch\_idx, x1, y1, x2, y2] or [num\_rois, 4] each line with format [x1, y1, x2, y2] - Constraints: > > > - Shape: 0 <= num\_rois <= num\_proposals > - Shape: shape(out[1])[1] must equal shape(in[0])[1] #### out[2..4] RoIs FPN : RoIs mapped to FPN levels [*roi\_min\_level* + 1, *roi\_max\_level*]. Note that each output may have a different shape since *num\_roi\_i* can vary for out[2..4]. This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: false - Data type: backend specific - Shape: [num\_rois\_i, 5] each line with format [batch\_idx, x1, y1, x2, y2] or [num\_rois\_i, 4] each line with format [x1, y1, x2, y2] - Constraints: > > > - Shape: 0 <= num\_rois\_i <= num\_proposals > - Datatype: Same datatype as out[1] > - Number: Number of out[2..4] provided must equal *roi\_max\_level* - *roi\_min\_level* > - Value: The sum of *num\_rois* across all *RoIs FPN* outputs must equal *num\_proposals* > - Shape: shape(out[2..4])[1] must equal shape(in[0])[1] ## ElementWiseAbs Computes absolute value of the input element-wise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_ABS](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a8ecc424dd6710f89683edaf12ce38e39) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseAdd Adds two tensors element-wise. The output is the sum of input tensors. out[0] = in[0] + in[1] Copy to clipboard Refer to ElementWiseAdd backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_ADD](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ad681988001e5f8ab73230a311f4ab034) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[1])[i] is dynamic then shape(in[0])[i] must be dynamic or must be compatible for broadcasting. #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Rank > 0 > - Must have same data format as in[0] (e.g. both sparse or both dense) > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(in[1])[i] must be dynamic or must be compatible for broadcasting. ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Must have same data format as in[0] (e.g. both sparse or both dense) > - Dynamic Shape: For each dimension, if both shape(in[0])[i] and shape(in[1])[i] are dynamic or if either shape(in[0])[i] or shape(in[1])[i] is dynamic and the other has a compatible dimension for broadcasting then shape(out[0])[i] must be dynamic. ## ElementWiseAnd Logical ANDs two tensors element-wise: out[0] = in[0] && in[1] Copy to clipboard where non-zero values are treated as true and zero as false. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_LOGICAL_AND](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a3e5d914633b5520c00b1668d2244b911) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] 1st input tensor - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank N #### in[1] 2nd input tensor - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank M - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## ElementWiseAsin Computes the arcsine of the input element-wise. Note that arcsine behavior is undefined for input values outside the range [-1, 1]. \[\mbox{out[0]} = \arcsin{(\mbox{in[0]})}\] References: - ONNX: [ops::Asin](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Asin) - PyTorch: [Asin](https://pytorch.org/docs/stable/generated/torch.asin.html#torch.asin) - TensorFlow: [asin](https://www.tensorflow.org/api_docs/python/tf/math/asin) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseAtan Computes the arctangent of the input element-wise. References: - ONNX: [ops::Atan](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Atan) - TensorFlow: [atan](https://www.tensorflow.org/api_docs/python/tf/math/atan) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseBinary Computes the specified binary *operation* on the input element-wise. Note that operations can be classified as being Numerical, Comparison, or Logical. Available element-wise operations: **ADD** - Adds two tensors element-wise. The output is the sum of input tensors. \[\mbox{out[0]} = \mbox{in[0]} + \mbox{in[1]}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_ADD](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ad681988001e5f8ab73230a311f4ab034) **AND** - Logical ANDs two tensors element-wise. \[\mbox{out[0]} = \mbox{in[0]}\ \&\&\ \mbox{in[1]}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_LOGICAL_AND](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a3e5d914633b5520c00b1668d2244b911) **DIVIDE** - Divides two tensors element-wise. The output is the result of dividing first input tensor by the second one. Result is undefined for any 0 value in the second input tensor. \[\mbox{out[0]} = \frac{\mbox{in[0]}}{\mbox{in[1]}}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_DIV](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a139794099b4137599bbc73af18b0d42a) **EQUAL** - Computes the equal logical operation element-wise on the two input tensors. \[\begin{split}\mbox{out[0]} = \begin{cases} \text{true}\ & \text{if}\ \mbox{in[0]} == \mbox{in[1]}, \\ \text{false}\ & \text{otherwise.} \end{cases}\end{split}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_EQUAL](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a4af31ebf58c6845d36fe26cf1a794d2c) **FLOOR\_DIV** - Divides two tensors element-wise and floor the result. Result is undefined for any 0 value in the second input tensor. Rounds towards the lowest integer. For Example: -2.3 rounds to -3 and 2.3 rounds to 2 \[\mbox{out[0]} = \text{floor}(\frac{\mbox{in[0]}}{\mbox{in[1]}})\] References: - TensorFlow: [tf.math.floordiv](https://www.tensorflow.org/api_docs/python/tf/math/floordiv) **FMOD** - Performs element-wise binary modulus. The sign of the remainder is the same as that of the dividend (in[0]). Note that behavior is undefined when elements from both in[0] and in[1] are 0. \[\mbox{out[0]} = \mbox{in[0]}\ \%\ \mbox{in[1]}\] References: - ONNX: [Mod](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Mod) **GREATER** - Computes the greater than logical operation element-wise on the input tensors. \[\begin{split}\mbox{out[0]} = \begin{cases} \text{true}\ & \text{if}\ \mbox{in[0]} > \mbox{in[1]}, \\ \text{false}\ & \text{otherwise.} \end{cases}\end{split}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_GREATER](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0afe61c7102e8fa16d19c2d7d8f4f15360) **GREATER\_EQUAL** - Computes the greater than or equal logical operation element-wise on the input tensors. \[\begin{split}\mbox{out[0]} = \begin{cases} \text{true}\ & \text{if}\ \mbox{in[0]}\ >=\ \mbox{in[1]}, \\ \text{false}\ & \text{otherwise.} \end{cases}\end{split}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_GREATER_EQUAL](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a3327f787f234f2a21f1fe14876ba85aa) **LESS** - Computes the less than logical operation element-wise on the input tensors. \[\begin{split}\mbox{out[0]} = \begin{cases} \text{true}\ & \text{if}\ \mbox{in[0]} < \mbox{in[1]}, \\ \text{false}\ & \text{otherwise.} \end{cases}\end{split}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_LESS](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a858bf52c76d31c41a8df611017a96db0) **LESS\_EQUAL** - Computes the less than or equal logical operation element-wise on the input tensors. \[\begin{split}\mbox{out[0]} = \begin{cases} \text{true}\ & \text{if}\ \mbox{in[0]}\ <=\ \mbox{in[1]}, \\ \text{false}\ & \text{otherwise.} \end{cases}\end{split}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_LESS_EQUAL](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0acf4d4d24685a6267d28f4873e7a43ae1) **MAXIMUM** - Element-wise maximum of two tensors. \[\mbox{out[0]} = \text{max}(\mbox{in[0]}, \mbox{in[1]})\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_MAXIMUM](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0aad00698dac694a7c1233dede8e8de91e) **MINIMUM** - Element-wise minimum of two tensors. \[\mbox{out[0]} = \text{min}(\mbox{in[0]}, \mbox{in[1]})\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_MINIMUM](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a8738794be11b7a43e5c83010f582e0a6) **MOD** - Performs element-wise binary modulus. The sign of the remainder is the same as that of the divisor (in[1]). Note that behavior is undefined when elements from both in[0] and in[1] are 0. This operation does not support floating point data types use FMOD instead. \[\mbox{out[0]} = \mbox{in[0]}\ \%\ \mbox{in[1]}\] References: - ONNX: [Mod](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Mod) **MULTIPLY** - Multiplies two tensors element-wise. The output is the result of multiplying first input tensor with the second one. \[\mbox{out[0]} = \mbox{in[0]} \times \mbox{in[1]}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_MUL](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ab34ca99890c827b536ce66256a803d7a) **NOT\_EQUAL** - Computes the not equal logical operation element-wise on the input tensors. \[\begin{split}\mbox{out[0]} = \begin{cases} \text{true}\ & \text{if}\ \mbox{in[0]}\ !=\ \mbox{in[1]}, \\ \text{false}\ & \text{otherwise.} \end{cases}\end{split}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_NOT_EQUAL](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a7b79220811dd60b3c9afaabdc6ea6842) **OR** - Logical ORs two tensors element-wise where non-zero values are treated as true and zero as false. \[\mbox{out[0]} = \mbox{in[0]}\ \vert \vert\ \mbox{in[1]}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_LOGICAL_OR](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a132ac3197242be5700d62b0037701718) **POWER** - Given base and exponent in input tensors, computes the (base^exponent) element-wise. \[\mbox{out[0]} = (\mbox{in[0]})^{\mbox{in[1]}}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_POW](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0af04a05fbde5ec0d2e4e088750a8451ad) - ONNX: [Pow](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Pow) **SQUARED\_DIFFERENCE** - Computes the element-wise difference between 2 tensors by subtracting the second tensor from the first and squares the results element-wise. \[\mbox{out[0]} = (\mbox{in[0]} - \mbox{in[1]}) \times (\mbox{in[0]} - \mbox{in[1]})\] References: - TensorFlow: [tf.math.squared_difference](https://www.tensorflow.org/api_docs/python/tf/math/squared_difference) **SUBTRACT** - Subtract two tensors element-wise. The output is the result of subtracting second input tensor from the first one. \[\mbox{out[0]} = \mbox{in[0]} - \mbox{in[1]}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_SUB](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a06a4248fe5ec71820ab95b87613780be) **XOR** - Logical XORs two tensors element-wise where non-zero values are treated as true and zero as false. \[\mbox{out[0]} = \mbox{in[0]}\ \text{^}\ \mbox{in[1]}\] References: - ONNX: [Xor](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Xor) - PyTorch: [torch.logical_xor](https://pytorch.org/docs/stable/generated/torch.asin.html#torch.logical_xor) - TensorFlow: [tf.math.logical_xor](https://www.tensorflow.org/api_docs/python/tf/math/logical_xor) Refer to ElementWiseBinary backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] Note that the data type supported depends on the classification of the *operation* selected and can be categorized into one of the following: - Numerical: BACKEND\_SPECIFIC - Comparison: BACKEND\_SPECIFIC - Logical: BACKEND\_SPECIFIC, QNN\_DATATYPE\_BOOL\_8 - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[1])[i] is dynamic then shape(in[0])[i] must be dynamic or must be compatible for broadcasting. #### in[1] Note that the data type supported depends on the classification of the *operation* selected and can be categorized into one of the following: - Numerical: BACKEND\_SPECIFIC - Comparison: BACKEND\_SPECIFIC - Logical: BACKEND\_SPECIFIC, QNN\_DATATYPE\_BOOL\_8 - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] for Numerical and Comparison operations with the exception of POWER. > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(in[1])[i] must be dynamic or must be compatible for broadcasting. ### Parameters #### operation Specifies the binary element-wise operation to use. Operations can be classified as one of the following: - Numerical operations: ADD, DIVIDE, FMOD, FLOOR\_DIV, MAXIMUM, MINIMUM, MOD, MULTIPLY, POWER, SQUARED\_DIFFERENCE, SUBTRACT - Comparison operations: EQUAL, GREATER, GREATER\_EQUAL, LESS, LESS\_EQUAL, NOT\_EQUAL - Logical operations: AND, OR, XOR - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Values: > > > - ADD = 0, > - AND = 1, > - DIVIDE = 2, > - EQUAL = 3, > - FLOOR\_DIV = 4, > - FMOD = 5, > - GREATER = 6, > - GREATER\_EQUAL = 7, > - LESS = 8, > - LESS\_EQUAL = 9, > - MAXIMUM = 10, > - MINIMUM = 11, > - MOD = 12, > - MULTIPLY = 13, > - NOT\_EQUAL = 14, > - OR = 15, > - POWER = 16, > - SQUARED\_DIFFERENCE = 17, > - SUBTRACT = 18, > - XOR = 19 ### Outputs #### out[0] Note that the data type supported depends on the classification of the *operation* selected and can be categorized into one of the following: - Numerical: BACKEND\_SPECIFIC - Comparison: BACKEND\_SPECIFIC, QNN\_DATATYPE\_BOOL\_8 - Logical: BACKEND\_SPECIFIC, QNN\_DATATYPE\_BOOL\_8 - Mandatory: true - Data type: backend specific - Shape: a tensor of Rank = max(N,M) - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] for Numerical operations > - Dynamic Shape: For each dimension, if both shape(in[0])[i] and shape(in[1])[i] are dynamic or if either shape(in[0])[i] or shape(in[1])[i] is dynamic and the other has a compatible dimension for broadcasting then shape(out[0])[i] must be dynamic. ## ElementWiseCeil Computes elementwise ceil on the input. Returns elementwise smallest integer not less than the input. References: - ONNX: [Ceil](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Ceil) - TensorFlow: [Ceil](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/math/ceil) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16, backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseCos Computes the cosine of the input element-wise. References: - TensorFlow: [tf.math.cos](https://www.tensorflow.org/api_docs/python/tf/math/cos) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16 - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseDivide Divides two tensors element-wise. The output is the result of dividing first input tensor by the second one. Result is undefined for any 0 value in the second input tensor. out[0] = in[0] / in[1] Copy to clipboard Refer to ElementWiseDivide backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_DIV](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a139794099b4137599bbc73af18b0d42a) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[1])[i] is dynamic then shape(in[0])[i] must be dynamic or must be compatible for broadcasting. #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(in[1])[i] must be dynamic or must be compatible for broadcasting. ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if both shape(in[0])[i] and shape(in[1])[i] are dynamic or if either shape(in[0])[i] or shape(in[1])[i] is dynamic and the other has a compatible dimension for broadcasting then shape(out[0])[i] must be dynamic. ## ElementWiseEqual Computes the equal logical operation elementwise on the two input tensors. out[0] = in[0] == in[1] Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_EQUAL](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a4af31ebf58c6845d36fe26cf1a794d2c) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## ElementWiseExp Computes exponential of input element-wise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_EXP](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a32bd40860ff7c3d91f5e62980bb52bc2) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16, backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseFloor Computes elementwise floor on the input. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_FLOOR](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0acdb4a57160153118dc6f87af0e4eccc5) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16, backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseFloorDiv Divides two tensors elementwise and floor the result. Result is undefined for any 0 value in the second input tensor. Rounds towards the lowest integer. For Example: -2.3 rounds to -3 and 2.3 rounds to 2 out[0] = floor(in[0] / in[1]) Copy to clipboard References: - TensorFlow: [tf.math.floordiv](https://www.tensorflow.org/api_docs/python/tf/math/floordiv) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Constraints: > > > - Datatype: Same datatype as in[0] ## ElementWiseFmod Performs element-wise binary modulus. The sign of the output elements is the same as elements in in[0]. Note that behavior is undefined when elements from both in[0] and in[1] are 0. \[\mbox{out[0]} = \mbox{in[0]} \ \% \ \mbox{in[1]}\] References: - ONNX: [Mod](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Mod) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Constraints: > > > - Datatype: Same datatype as in[0] ## ElementWiseGreater Computes the greater than logical operation elementwise on the input tensors. out[0] = in[0] > in[1] Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_GREATER](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0afe61c7102e8fa16d19c2d7d8f4f15360) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## ElementWiseGreaterEqual Computes the greater than or equal logical operation elementwise on the input tensors. out[0] = in[0] >= in[1] Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_GREATER_EQUAL](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a3327f787f234f2a21f1fe14876ba85aa) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## ElementWiseLess Computes the less than logical operation elementwise on the input tensors. out[0] = in[0] < in[1] Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_LESS](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a858bf52c76d31c41a8df611017a96db0) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## ElementWiseLessEqual Computes the less than or equal logical operation elementwise on the input tensors. Tensors must have same rank and tensor dimensions must be compatible. Two dimensions are compatible when they are equal or in[1] dimension is 1 (i.e. in[1] is broadcast across in[0]). out[0] = in[0] <= in[1] Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_LESS_EQUAL](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0acf4d4d24685a6267d28f4873e7a43ae1) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## ElementWiseLog Computes logarithm of input element-wise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_LOG](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ad22137e675fafa6d72da7b52952cfb78) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16 - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseMaximum Element-wise maximum of two tensors. out[0] = max(in[0], in[1]) Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_MAXIMUM](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0aad00698dac694a7c1233dede8e8de91e) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Constraints: > > > - Datatype: Same datatype as in[0] ## ElementWiseMinimum Element-wise minimum of two tensors. out[0] = min(in[0], in[1]) Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_MINIMUM](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a8738794be11b7a43e5c83010f582e0a6) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Constraints: > > > - Datatype: Same datatype as in[0] ## ElementWiseMod Performs element-wise binary modulus. The sign of the output elements is the same as elements in in[1]. Note that behavior is undefined when elements from both in[0] and in[1] are 0. \[\mbox{out[0]} = \mbox{in[0]} \ \% \ \mbox{in[1]}\] References: - ONNX: [Mod](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Mod) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Does not support floating point data types #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Constraints: > > > - Datatype: Same datatype as in[0] ## ElementWiseMultiply Multiplies two tensors element-wise. The output is the result of multiplying first input tensor with the second one. out[0] = in[0] * in[1] Copy to clipboard Refer to ElementWiseMultiply backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_MUL](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ab34ca99890c827b536ce66256a803d7a) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[1])[i] is dynamic then shape(in[0])[i] must be dynamic or must be compatible for broadcasting. #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(in[1])[i] must be dynamic or must be compatible for broadcasting. ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if both shape(in[0])[i] and shape(in[1])[i] are dynamic or if either shape(in[0])[i] or shape(in[1])[i] is dynamic and the other has a compatible dimension for broadcasting then shape(out[0])[i] must be dynamic. ## ElementWiseNeg Computes numerical negative of input element-wise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_NEG](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a3418665f9871dca43e3cb2efcac3990b) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseNeuron Computes the specified *operation* on the input element-wise. Available element-wise operations: **Elu** - Computes the Exponential Linear Unit operation. \[\mbox{out[0]} = \max(0, \mbox{in[0]}) + \min(0, \alpha \times (e^{\mbox{in[0]}} - 1))\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_ELU](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a043a2da1768dd3441810b2033ededf29) - ONNX: [Elu](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Elu) **Gelu** - Computes the Gaussian error linear unit operation. \[\mbox{out[0]} = \frac{\mbox{in[0]}}{2} \times (1 + \text{erf} (\frac{\mbox{in[0]}}{\sqrt{2}}))\] References: - TensorFlow: [tf.keras.activations.gelu](https://www.tensorflow.org/api_docs/python/tf/keras/activations/gelu) **HardSigmoid** - Computes the HardSigmoid function element-wise on the input. \[\mbox{out[0]} = \max(0, \min(1, \alpha \times \mbox{in[0]} + \beta))\] References: - ONNX: [HardSigmoid](https://github.com/onnx/onnx/blob/main/docs/Operators.md#hardsigmoid) **HardSwish** - Computes the HardSwish operation. \[\mbox{out[0]} = \frac{\mbox{in[0]} \times \max(0, \min(6, \mbox{in[0]} + 3))}{6}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_HARD_SWISH](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ac88e58ac12cdd6ef3e20d7204ec6f33e) - TensorFlow: [hard_swish](https://github.com/tensorflow/models/blob/master/official/modeling/activations/swish.py) **Relu** - Computes the Rectified Linear Unit operation. \[\mbox{out[0]} = \max(0, \mbox{in[0]})\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_RELU](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0abb2f979866b131c5089ba0caaecee656) - TensorFlow: [tf.nn.relu](https://www.tensorflow.org/api_docs/python/tf/nn/relu) **ReluMinMax** - Computes the Rectified Linear Unit Min Max operation. \[\mbox{out[0]} = \min(\text{max\_value}, \max(\text{min\_value}, \mbox{in[0]}))\] where min\_value <= max\_value. References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_RELU6](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a04a24c2d6f0aac4c3f5324c1d7764714) - TensorFlow: [tf.nn.relu6](https://www.tensorflow.org/api_docs/python/tf/nn/relu6) **Sigmoid** - Computes the sigmoid activation function element-wise. \[\mbox{out[0]} = \frac{1}{1 + e^{(-1 \times \mbox{in[0]})}}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_LOGISTIC](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a82a340eb540933f638db420369650483) **Softplus** - Computes the softplus function to the input tensor element-wise. Note that when \((\mbox{in[0]} \times \beta)\) > *threshold* the implementation reverts to a linear function to preserve numerical stability. \[\begin{split}\mbox{out[0]} = \begin{cases} \mbox{in[0]} & \text{if}\ \mbox{in[0]} \times \beta > \text{threshold}, \\ \frac{1}{\beta} \times \ln{(e^{(\beta \times \mbox{in[0]})} + 1)} & \text{otherwise} \end{cases}\end{split}\] References: - ONNX: [Softplus](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Softplus) - PyTorch: [torch.nn.Softplus](https://pytorch.org/docs/stable/generated/torch.nn.Softplus.html) - TensorFlow: [tf.math.softplus](https://www.tensorflow.org/api_docs/python/tf/math/softplus) **Tanh** - Computes the hyperbolic tangent function element-wise. \[\mbox{out[0]} = \tanh{(\mbox{in[0]})}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_TANH](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a4b63c9caab823f112d82d853a77381e5) Refer to ElementWiseNeuron backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### operation Specifies the element-wise operation to use. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Values: > > > - ELU = 0, > - GELU = 1, > - HARD\_SIGMOID = 2, > - HARD\_SWISH = 3, > - RELU = 4, > - RELU\_MIN\_MAX = 5, > - SIGMOID = 6, > - SOFTPLUS = 7, > - TANH = 8 #### alpha (\(\alpha\)) The alpha (\(\alpha\)) value for Elu and HardSigmoid function. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: > > > - 1.0 for QNN\_OP\_ELEMENT\_WISE\_NEURON\_OPERATION\_ELU > - 0.2 for QNN\_OP\_ELEMENT\_WISE\_NEURON\_OPERATION\_HARD\_SIGMOID - Constraints: > > > - Must have *operation* set to ELU or HARD\_SIGMOID for this parameter to be valid. #### beta (\(\beta\)) The beta (\(\beta\)) value for HardSigmoid and Softplus functions. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: > > > - 0.5 for QNN\_OP\_ELEMENT\_WISE\_NEURON\_OPERATION\_HARD\_SIGMOID > - 1.0 for QNN\_OP\_ELEMENT\_WISE\_NEURON\_OPERATION\_SOFTPLUS - Constraints: > > > - Must have *operation* set to HARD\_SIGMOID or SOFTPLUS for this parameter to be valid. > - Value: beta provided for SOFTPLUS must be > 0. #### min\_value The minimum value in operation RELU\_MIN\_MAX - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: None - Constraints: > > > - Must have *operation* set to RELU\_MIN\_MAX for this parameter to be valid. > - Must be provided when *operation* is set to RELU\_MIN\_MAX. > - Value: min\_value must be <= max\_value #### max\_value The maximum value in operation RELU\_MIN\_MAX - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: None - Constraints: > > > - Must have *operation* set to RELU\_MIN\_MAX for this parameter to be valid. > - Must be provided when *operation* is set to RELU\_MIN\_MAX. #### threshold Values above the threshold revert to a linear function. Note that parameter is disabled by default or when set to a negative. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: -1.0 - Constraints: > > > - Must have *operation* set to SOFTPLUS for this parameter to be valid. ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(out[0])[i] must be dynamic. ## ElementWiseNot Logical NOT of a tensor element-wise: out[0] = 1 if in[0] == 0 out[0] = 0 otherwise Copy to clipboard where non-zero values are treated as true and zero as false. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_LOGICAL_NOT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0acb90aff5615d001708e9e4c6545b2b3c) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseNotEqual Computes the not equal logical operation elementwise on the input tensors. out[0] = in[0] != in[1] Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_NOT_EQUAL](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a7b79220811dd60b3c9afaabdc6ea6842) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## ElementWiseOr Logical ORs two tensors element-wise: out[0] = in[0] || in[1] Copy to clipboard where non-zero values are treated as true and zero as false. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_LOGICAL_OR](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a132ac3197242be5700d62b0037701718) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] 1st input tensor - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank N #### in[1] 2nd input tensor - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: A Tensor of rank M ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## ElementWisePower Given base and exponent in input tensors, computes the (base^exponent) element-wise. out[0] = in[0] ^ in[1] Copy to clipboard Refer to ElementWisePower backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_POW](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0af04a05fbde5ec0d2e4e088750a8451ad) - ONNX: [Pow](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Pow) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] Tensor specifying the base. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[1])[i] is dynamic then shape(in[0])[i] must be dynamic or must be compatible for broadcasting. #### in[1] Tensor specifying the exponent. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(in[1])[i] must be dynamic or must be compatible for broadcasting. ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Shape: a tensor of rank = max(N,M) - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if both shape(in[0])[i] and shape(in[1])[i] are dynamic or if either shape(in[0])[i] or shape(in[1])[i] is dynamic and the other has a compatible dimension for broadcasting then shape(out[0])[i] must be dynamic. ## ElementWiseRound Computes elementwise rounding on the input. The operation rounds the values in the input to the nearest integer. Halfs are rounded to the nearest even integer. E.g: 2.5 is rounded to 2.0; -3.5 is rounded to -4.0. References: - ONNX: [Round](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Round) - TensorFlow: [Round](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/math/round) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16, backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseRsqrt Computes reciprocal of the square root of the input tensor element-wise. Negative elements are unsupported. If an element is negative, behaviour is undefined. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_RSQRT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ac48dc0e528086e4d241d42d2923467da) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16 - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseSelect Given a boolean condition tensor, select value in first input tensor (if true) or value in second input tensor (if false). Note that the three input tensors must be either of the same shape or be able to broadcast to a common shape. out[0] = in[0] ? in[1] : in[2] Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_SELECT](https://developer.android.com/ndk/reference/group/neural-networks.html#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a49b2dc37ea9219789a6d82f281499dbb) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] condition input - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Shape: Rank > 0 #### in[2] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank K - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N, M, K) ## ElementWiseSign Computes the sign of the input tensor element-wise. \[\begin{split}\mbox{out[0]} = \text{Sign}(\mbox{in[0]}) = \begin{cases} 1 & \text{if}\ \mbox{in[0]} > 0, \\ 0 & \text{if}\ \mbox{in[0]} = 0, \\ -1 & \text{otherwise} \end{cases}\end{split}\] References: - ONNX: [ops::Sign](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Sign) - PyTorch: [Sign](https://pytorch.org/docs/stable/generated/torch.sign.html) - TensorFlow: [Sign](https://www.tensorflow.org/api_docs/python/tf/math/sign) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseSin Computes the sin of the input element-wise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_SIN](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0af16ba8e2a692bce91fb53c0cd64eed11) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16 - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseSoftplus Applies the softplus function to the input tensor element-wise to produce an output tensor. \[\mbox{out[0]} = \frac{1}{\beta} \* \ln{(e^{\beta \* \mbox{in[0]}} + 1)}\] Note that when in[0] \* \(\beta\) > *threshold* the implementation reverts to a linear function to preserve numerical stability. References: - ONNX: [ops::Softplus](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Softplus) - Pytorch: [Softplus](https://pytorch.org/docs/stable/generated/torch.nn.Softplus.html) - TensorFlow: [Softplus](https://www.tensorflow.org/api_docs/python/tf/math/softplus) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters #### beta (\(\beta\)) The beta (\(\beta\)) value for the Softplus formulation. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1.0 - Constraints: > > > - Value: beta must be > 0 #### threshold Values above the threshold revert to a linear function. Note that parameter is disabled by default or when set to a negative. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: -1.0 ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ElementWiseSquaredDifference Computes the element-wise difference between 2 tensors by subtracting the second tensor from the first and squares the results element-wise. out[0] = (in[0] - in[1]) * (in[0] - in[1]) Copy to clipboard References: - TensorFlow: [squared_difference](https://www.tensorflow.org/api_docs/python/tf/math/squared_difference) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Constraints: > > > - Datatype: Same datatype as in[0] ## ElementWiseSquareRoot Computes the square root of the input tensor element-wise. Negative elements are unsupported. If an element is negative, behaviour is undefined. Refer to ElementWiseSquareRoot backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_SQRT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0acf8329b28ec9fd0aa9d0e6a41aaba628) - ONNX: [Sqrt](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Sqrt) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(out[0])[i] must be dynamic. ## ElementWiseSubtract Subtract two tensors element-wise. The output is the result of subtracting second input tensor from the first one. out[0] = in[0] - in[1] Copy to clipboard Refer to ElementWiseSubtract backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_SUB](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a06a4248fe5ec71820ab95b87613780be) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[1])[i] is dynamic then shape(in[0])[i] must be dynamic or must be compatible for broadcasting. #### in[1] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(in[1])[i] must be dynamic or must be compatible for broadcasting. ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = max(N,M) - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if both shape(in[0])[i] and shape(in[1])[i] are dynamic or if either shape(in[0])[i] or shape(in[1])[i] is dynamic and the other has a compatible dimension for broadcasting then shape(out[0])[i] must be dynamic. ## ElementWiseUnary Computes the specified unary *operation* on the input element-wise. Note that operations can be classified as being Numerical or Logical. Available element-wise operations: **ABS** - Computes the absolute value element-wise. \[\mbox{out[0]} = \lvert \mbox{in[0]} \rvert\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_ABS](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a8ecc424dd6710f89683edaf12ce38e39) **ASIN** - Computes the arcsine element-wise. Note that if elements are outside the range [-1, 1] behavior is undefined. \[\mbox{out[0]} = \arcsin{(\mbox{in[0]})}\] References: - ONNX: [Asin](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Asin) - PyTorch: [torch.asin](https://pytorch.org/docs/stable/generated/torch.asin.html#torch.asin) - TensorFlow: [tf.math.asin](https://www.tensorflow.org/api_docs/python/tf/math/asin) **ATAN** - Computes the arctangent element-wise. \[\mbox{out[0]} = \arctan{(\mbox{in[0]})}\] References: - ONNX: [Atan](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Atan) - TensorFlow: [tf.math.atan](https://www.tensorflow.org/api_docs/python/tf/math/atan) **CEIL** - Applies ceil() on the input element-wise. E.g. ceil([-1.5, 1.2, 2.0]) will output [-1.0, 2.0, 2.0]. \[\mbox{out[0]} = \text{ceil}(\mbox{in[0]})\] References: - ONNX: [Ceil](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Ceil) - TensorFlow: [tf.math.ceil](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/math/ceil) **COS** - Computes the cosine of the input element-wise. \[\mbox{out[0]} = \cos{(\mbox{in[0]})}\] References: - TensorFlow: [tf.math.cos](https://www.tensorflow.org/api_docs/python/tf/math/cos) **EXP** - Computes the exponential of the input element-wise. \[\mbox{out[0]} = e^{(\mbox{in[0]})}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_EXP](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a32bd40860ff7c3d91f5e62980bb52bc2) **FLOOR** - Applies floor() on the input element-wise. E.g. floor([-1.5, 1.2, 2.0]) will output [-2.0, 1.0, 2.0]. \[\mbox{out[0]} = \text{floor}(\mbox{in[0]})\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_FLOOR](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0acdb4a57160153118dc6f87af0e4eccc5) **LOG** - Computes the logarithm of the input element-wise. Note that if an element is 0 behavior is undefined. \[\mbox{out[0]} = \log{(\mbox{in[0]})}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_LOG](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ad22137e675fafa6d72da7b52952cfb78) **NEG** - Computes the numerical negative of the input element-wise. \[\mbox{out[0]} = -1 \times \mbox{in[0]}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_NEG](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a3418665f9871dca43e3cb2efcac3990b) **NOT** - Applies logical NOT on the input element-wise. \[\begin{split}\mbox{out[0]} = \text{Not}(\mbox{in[0]}) = \begin{cases} 1 & \text{if}\ \mbox{in[0]} = 0, \\ 0 & \text{otherwise.} \end{cases}\end{split}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_LOGICAL_NOT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0acb90aff5615d001708e9e4c6545b2b3c) **RECIPROCAL** - Computes the reciprocal of the input element-wise. Note that if an element is 0 behavior is undefined. \[\mbox{out[0]} = \frac{1}{\mbox{in[0]}}\] References: - ONNX: [Reciprocal](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Reciprocal) - PyTorch: [torch.reciprocal](https://pytorch.org/docs/stable/generated/torch.reciprocal.html) - TensorFlow: [tf.math.reciprocal](https://www.tensorflow.org/api_docs/python/tf/math/reciprocal) **ROUND** - Rounds to the nearest integer element-wise. Note that elements at halves are rounded to the nearest even number. E.g. round([2.5, -4.5, 1.5]) will output [2.0, -4.0, 2.0]. \[\mbox{out[0]} = \text{round}(\mbox{in[0]})\] References: - ONNX: [Round](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Round) - TensorFlow: [tf.math.round](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/math/round) **RSQRT** - Computes the reciprocal of the square root element-wise. Negative elements are unsupported. If an element is negative or 0, behavior is undefined. \[\mbox{out[0]} = \frac{1}{\sqrt{\mbox{in[0]}}}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_RSQRT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ac48dc0e528086e4d241d42d2923467da) **SIGN** - Computes the sign of the input element-wise. \[\begin{split}\mbox{out[0]} = \text{sign}(\mbox{in[0]}) = \begin{cases} 1 & \text{if}\ \mbox{in[0]} > 0, \\ 0 & \text{if}\ \mbox{in[0]} = 0, \\ -1 & \text{otherwise.} \end{cases}\end{split}\] References: - ONNX: [Sign](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Sign) - PyTorch: [torch.sign](https://pytorch.org/docs/stable/generated/torch.sign.html) - TensorFlow: [tf.math.sign](https://www.tensorflow.org/api_docs/python/tf/math/sign) **SIN** - Computes the sine of the input element-wise. \[\mbox{out[0]} = \sin{(\mbox{in[0]})}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_SIN](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0af16ba8e2a692bce91fb53c0cd64eed11) **SQRT** - Computes the square root of the input element-wise. Negative elements are unsupported. If an element is negative, behavior is undefined. \[\mbox{out[0]} = \sqrt{\mbox{in[0]}}\] References: - Android NDK NeuralNetWorks: [ANEURALNETWORKS_SQRT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0acf8329b28ec9fd0aa9d0e6a41aaba628) - ONNX: [Sqrt](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Sqrt) Refer to ElementWiseUnary backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] Note that the data type supported depends on the classification of the *operation* selected and can be categorized into one of the following: - Numerical: BACKEND\_SPECIFIC - Logical: BACKEND\_SPECIFIC, QNN\_DATATYPE\_BOOL\_8 See *operation* parameter for classification per operation. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### operation Specifies the unary element-wise operation to use. Operations can be classified as one of the following: - Numerical operations: ABS, ASIN, ATAN, CEIL, COS, EXP, FLOOR, LOG, NEG, RECIPROCAL, ROUND, RSQRT, SIGN, SIN, and SQRT. - Logical operations: NOT. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Values: > > > - ABS = 0, > - ASIN = 1, > - ATAN = 2, > - CEIL = 3, > - COS = 4, > - EXP = 5, > - FLOOR = 6, > - LOG = 7, > - NEG = 8, > - NOT = 9, > - RECIPROCAL = 10, > - ROUND = 11, > - RSQRT = 12, > - SIGN = 13, > - SIN = 14, > - SQRT = 15 ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(out[0])[i] must be dynamic. ## ElementWiseXor Logical XOR operation elementwise on two input tensors: out[0] = in[0] ^ in[1] Copy to clipboard where non-zero values are treated as true and zero as false. References: - ONNX: [Xor](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Xor) - PyTorch: [torch.logical_xor](https://pytorch.org/docs/stable/generated/torch.asin.html#torch.logical_xor) - TensorFlow: [tf.math.logical_xor](https://www.tensorflow.org/api_docs/python/tf/math/logical_xor) - QNN: [ElementwiseRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) - QNN: [RankMatchingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] 1st input tensor - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank N #### in[1] 2nd input tensor - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: A Tensor of rank M ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8, backend specific - Shape: a tensor of rank = max(N,M) ## Elu The Exponential Linear Unit operation computes: output= max(0, input) + min(0, alpha * (exp(input) - 1)) Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_ELU](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a043a2da1768dd3441810b2033ededf29) - ONNX: [ops::Elu](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Elu) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters #### alpha Alpha parameter - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1.0 ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ExpandDims This operation inserts a dimension of size 1 at the dimension index *axis* or dimension indices *axes* of *in[0]* tensor. The number of elements in output tensor remains the same as in input tensor. This functionality can also be achieved using the [Reshape](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#reshape) operation. Refer to ExpandDims backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_EXPAND_DIMS](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a2ac3c3daa1904f202589bef8c1c1860a) - TensorFlow: [expand_dims](https://www.tensorflow.org/api_docs/python/tf/expand_dims) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank N > 0 ### Parameters #### axis Scalar index specifying the dimension to insert a value of 1. Both scalar *axis* and its tensor counterpart *axes* are optional, but at least one of them must be provided. However, if *axes* is non-empty, the value of *axis* is ignored. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Constraints: > > > - Value: Must be in range [0,M) #### axes Indices specifying the dimensions to insert a value of 1. Both scalar *axis* and its tensor counterpart *axes* are optional, but at least one of them must be provided. When both are specified, values of *axes* take precedence over scalar *axis*. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [K] - Default: {0,..,0} - Constraints: > > > - Value: Must be in range [0, M) > - Value: Must be unique ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M, where M = N + 1 if *axis* is used. Otherwise, M = N + K. - Dynamic Shape: All dimensions other than the inserted dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] other than the expanded dimensions > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then the corresponding dimension of out[0] after being expanded must be dynamic. ## ExtractGlimpse Extracts a set of windows called glimpses from the input tensor. If the window only partially overlaps the input, the non-overlapping areas will be filled with noise. References: - TensorFlow: [ops::ExtractGlimpse](https://www.tensorflow.org/api_docs/python/tf/image/extract_glimpse) ### Inputs #### in[0] Input data - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel] #### in[1] Glimpse offsets - If *normalized* and *centered*, the offsets point to the center of the glimpse. Otherwise, the offsets point to the upper-left of the glimpse. - Mandatory: true - Data type: backend specific - Shape: [batch, 2] each batch with format [y, x] ### Parameters #### size - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [2] with format [glimpse\_height, glimpse\_width] #### centered If true, offset coordinates are relative to the center of image. Otherwise, offset coordinates are relative to the upper-left corner of the image. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 #### normalized If not *normalized* and not *centered*, the *offsets* are treated as the number of pixels from the upper-left of the image. - (0.0,0.0) corresponds to the image upper-left corner. - (height - 1.0,width - 1.0) corresponds to the image bottom-right corner. If not *normalized* and *centered*, the *offsets* are treated as the number of pixels from the center of the image. - (0.0,0.0) corresponds to the image center. - (height / 2.0,width / 2.0) corresponds to the image bottom-right corner. If *normalized* and not *centered*, normalized coordinates are in the range [0.0,1.0] and correspond to the minimum and maximum of each height and width. - (0.0,0.0) corresponds to the image upper-left corner. - (0.5,0.5) corresponds to the image center. - and (1.0,1.0) corresponds to the image bottom-right corner. If *normalized* and *centered*, normalized coordinates are in the range [-1.0,1.0] where - (-1.0,-1.0) corresponds to the image upper-left corner. - (0.0,0.0) corresponds to the image center. - and (1.0,1.0) corresponds to the image bottom-right corner. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 #### noise Indicates the noise distribution. - 0: Uniform - 1: Gaussian - 2: Zeroes - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Default: 0 - Values: > > > - Uniform = 0, > - Gaussian = 1, > - Zeroes = 2 ### Outputs #### out[0] Output data - Mandatory: true - Data type: backend specific - Shape: [batch, glimpse\_height, glimpse\_width, channel] - Constraints: > > > - Datatype: Same datatype as in[0] ## ExtractPatches Extracts patches from the input image in[0]. Patches are of shape *sizes* and are *strides* apart in the input image. All extracted patches are flattened and stacked in the *channel\_out* dimension of out[0]. References: - TensorFlow: [ops::ExtractPatches](https://www.tensorflow.org/api_docs/python/tf/image/extract_patches) ### Inputs #### in[0] Input data - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel\_in] ### Parameters #### size The size of the extracted patches. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [height\_size, width\_size] - Constraints: > > > - Value: Sizes must be > 0 #### stride Determines how far the centers of two consecutive patches are in the images. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### rate Specifies how far two consecutive patch samples are in the input. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [height\_rate, width\_rate] - Constraints: > > > - Value: Rates must be > 0 #### padding When set to QNN\_OP\_EXTRACT\_PATCHES\_PADDING\_VALID: Only patches which are fully contained in the input image are included. When set to QNN\_OP\_EXTRACT\_PATCHES\_PADDING\_SAME: All patches with starting points inside the input are included and areas outside the input default to zero. Note that padding has no effect on the size of each patch and only determines how many patches are extracted. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Values: > > > - VALID = 0, > - SAME = 1 ### Outputs #### out[0] Output patches : contains image patches of size height\_size \* width\_size \* channel\_in flattened in the channel\_out dimension. Spatial dimensions height\_out and width\_out are functions of *sizes*, *strides*, *rates*, and *padding*. // When padding is set to QNN_OP_EXTRACT_PATCHES_PADDING_VALID dilated_height = size[0] + (size[0] - 1) * (rate[0] - 1) dilated_width = size[1] + (size[1] - 1) * (rate[1] - 1) shape(out[0])[height_out] = floor((shape(in[0])[height] - dilated_height) / stride[0] + 1) shape(out[0])[width_out] = floor((shape(in[0])[width] - dilated_width) / stride[1] + 1) // When padding is set to QNN_OP_EXTRACT_PATCHES_PADDING_SAME shape(out[0])[height_out] = ceil(shape(in[0])[height] / stride[0]) shape(out[0])[width_out] = ceil(shape(in[0])[width] / stride[1]) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, height\_out, width\_out, channel\_out], where channel\_out = height\_size \* width\_size \* channel\_in. - Constraints: > > > - Datatype: Same datatype as in[0] ## FullyConnected The FullyConnected operation connects the all input elements with each output element through weights and biases. The weights tensor has shape [m, n] where n is the number of input elements and m is the units of weights, output and optional biases. The input activation must be reshapable to [batch, n] (see [Reshape](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#reshape) operation definition) and the operation computes mathematically: outputVector = ( inputAsVector * weightsMatrix ) + biasesVector Copy to clipboard for one batch of input, where \* denotes matrix multiply operation. Refer to FullyConnected backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_FULLY_CONNECTED](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0aaada7a3dbaf4676aba560c933ff610c5) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [n] or Rank >= 2 reshapable to [batch, n] - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 > - Shape: Rank >= 2 must be reshapable to [batch, n] #### in[1] weights - Mandatory: true - Data type: backend specific - Shape: [m, n] - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: if rank(in[0]) = 1, then shape(in[0])[n] and shape(in[1])[n] must be both dynamic or both static. Otherwise if rank(in[0]) > 1, shape(in[1])[n] must be static. #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [m] - Dynamic Shape: All dimensions can be dynamic. - Default: [0] - Constraints: > > > - Dynamic Shape: shape(in[1])[m] and shape(in[2])[m] must be both dynamic or both static. ### Parameters #### keep\_dims If true, the rank of in[0] and out[0] will remain the same, and all but the last dimension will be equal in shape. For dimensions to be preserved, the product of the batch dimensions of in[0] (all but the last dimension) must be equal to batch, defined by in[0] above. This is because: total # of outputs = (total # of batches) \* m Since the total # of outputs and m are the same regardless of *keep\_dims*, the total # of batches must remain the same as well. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: If the rank of in[0] is 1: [m]. If the rank of in[0] is > 1: [batch, m], unless *keep\_dims* is true, then […, m] where … is all but the last dimension of in[0] - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: If rank(in[0]) > 1 and keep\_dims is set to false and any dimension for in[0] is dynamic then shape(out[0])[batch] must be dynamic. Otherwise, if rank(in[0]) > 1 and keep\_dims is set to true, for each dimension with the exemption of the last dimension, if shape(in[0])[i] is dynamic then shape(out[0])[i] must be dynamic. > - Dynamic Shape: If shape(in[1])[m] is dynamic then shape(out[0])[m] must be dynamic. ## Gather Gather input data from the specified axis and indices. Note that for indices generated from other Operations (e.g. NonZero) we permit -1 to be provided as a value to indicate an index for Gather Operation to skip/ignore. Refer to Gather backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_GATHER](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a4bcb358f8c49e45e386a0a75405d5763) - ONNX: [ops::Gather](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Gather) ### Inputs #### in[0] Input activation, also known as table - Mandatory: true - Data type: backend specific - Shape: a tensor of rank n - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 #### in[1] Indices in in[0] to extract based on axis. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32, backend specific - Shape: a tensor of rank k - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Value: Indices must be in range [0, shape(in[0])[*axis*] - 1] > - Shape: Rank > 0 ### Parameters #### axis axis : The axis to gather. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Constraints: > > > - Value: Must be in range [0, n-1] ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = n + k - 1 - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[1])[i] is dynamic then shape(out[0])[axis + i] must be dynamic. For all other dimensions except the axis dimension, if shape(in[0])[i] is dynamic then the corresponding dimension of out[0] must be dynamic. ## GatherElements Gather input data from the specified axis and indices, where each index value pertains to a single value from the input data. The following example demonstrates how the output is produced in a 3D case (*n* = 3): output[i][j][k] = input[index[i][j][k]][j][k] (if axis = 0) output[i][j][k] = input[i][index[i][j][k]][k] (if axis = 1) output[i][j][k] = input[i][j][index[i][j][k]] (if axis = 2) Copy to clipboard Note that for indices generated from other Operations (e.g. NonZero) we permit -1 to be provided as a value to indicate an index for GatherElements Operation to skip/ignore. References: - ONNX: [ops::GatherElements](https://github.com/onnx/onnx/blob/master/docs/Operators.md#GatherElements) ### Inputs #### in[0] Input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank n - Constraints: > > > - Shape: Rank > 0 #### in[1] Index tensor : contains indices in in[0] to extract based on axis. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32 - Shape: a tensor of rank n - Constraints: > > > - Value: Indices must be in range [0, shape(in[0])[*axis*] - 1] ### Parameters #### axis axis : The axis of in[0] to gather on. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Constraints: > > > - Value: Must be in range [0, n-1] ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank n - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[1] ## GatherNd Gather slices of input data from the specified indices. This is similar to Gather operation, where indices define slices into the first dimension of the input data. In GatherNd, indices define slices into the first N dimensions of the input data, where N = Shape(in[1])[k-1]. Note that for indices generated from other Operations (e.g. NonZero) we permit -1 to be provided as a value to indicate an index for GatherNd Operation to skip/ignore. References: - ONNX: [ops::GatherND](https://github.com/onnx/onnx/blob/master/docs/Operators.md#GatherND) - TensorFlow: [gather_nd](https://www.tensorflow.org/api_docs/python/tf/gather_nd) ### Inputs #### in[0] Input activation, also known as table - Mandatory: true - Data type: backend specific - Shape: a tensor of rank n - Constraints: > > > - Shape: Rank > 0 #### in[1] Indices in in[0] to extract. Note that when Shape(in[1])[k-1] is equal to *n*, the indices provided index the full rank of in[0] and gather scalars. Otherwise, when Shape(in[1])[k-1] is less than *n*, the indices provided refer to row slices of in[0] to gather. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32, backend specific - Shape: a tensor of rank k - Constraints: > > > - Shape: Shape(in[1])[k-1] must be > 0 and <= (*n* - *batch\_dims*) > - Value: Indices must be non-negative and within the range of the corresponding dimension of in[0] > - Shape: Rank > 0 ### Parameters #### batch\_dims The number of batch dimensions. Note that gather starts indexing from dimension in[0][batch\_dims:]. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Constraints: > > > - Value: batch\_dims < min(*n*, *k*) > - Shape: Shape(in[0])[:batch\_dims] must equal Shape(in[1])[:batch\_dims] ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank = n + k - Shape(in[1])[k-1] - *batch\_dims* - 1, where *n* is the rank of in[0] and *k* is the rank of in[1]. - Constraints: > > > - Datatype: Same datatype as in[0] ## Gelu The **G**aussian **e**rror **l**inear **u**nit operation computes: out[0] = in[0]/2 * (1 + erf(in[0] / sqrt(2))) Copy to clipboard References: - TensorFlow: [ops::gelu](https://www.tensorflow.org/api_docs/python/tf/keras/activations/gelu) ### Inputs #### in[0] Input activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] Output activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## GenerateProposals Generates bounding-box proposals from an input feature-map by applying a transform to a set of predefined bounding-box anchors. The number of proposals is then limited by applying hard non-max suppression (NMS). References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_GENERATE_PROPOSALS](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a72484020f2c41c814de0a7bf93dbbfd4) ### Inputs #### in[0] Input feature map - Mandatory: true - Data type: backend specific - Shape: [batch,height,width,num\_anchors] #### in[1] Transform tensor. Elements can be considered 4-tuples of (dx,dy,dw,dh), as described in the reference. - Mandatory: true - Data type: backend specific - Shape: [batch,height,width,num\_anchors\*4] #### in[2] Anchor tensor. Elements can be considered 4-tuples of [x1,y1,x2,y2] coordinates in the original image. - Mandatory: true - Data type: backend specific - Shape: [num\_anchors,4] - Constraints: > > > - Datatype: Same datatype as in[0] #### in[3] Image sizes tensor. Elements should be interpreted as [image\_height,image\_width] for each image in the batch. - Mandatory: true - Data type: backend specific - Shape: [batch,2] - Constraints: > > > - Datatype: Same datatype as in[2] ### Parameters #### img\_size\_ratio Gives the ratio between the original image and the feature map, in the form [height\_ratio, width\_ratio] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: [2] with elements [height\_ratio, width\_ratio] #### min\_size Sets a minimum size for boxes before applying NMS. Boxes with width or height smaller than this value will be filtered out. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0 #### pre\_nms\_limit Sets a maximum number of boxes before applying NMS. The boxes with the lowest scores will be dropped to achieve this limit. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: No limit is applied #### post\_nms\_limit Sets a maximum number of boxes after applying NMS. The boxes with the lowest scores will be dropped to achieve this limit. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: No limit is applied #### iou\_threshold IoU threshold for the NMS operation - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar #### bbox\_xform\_clip When set to true bounding box are clipped to minimum bounding box height and width after transformation. Otherwise, no clipping is done. bbox_xform_clip_value = log(1000.0 / 16.0) dh = min(dh, bbox_xform_clip_value) dw = min(dw, bbox_xform_clip_value) Copy to clipboard - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Gives the score for each bounding box. Boxes corresponding to a given input batch element are grouped contiguously. - Mandatory: true - Data type: backend specific - Shape: [num\_boxes]. Max and current dimension value for *num\_boxes* may differ, and current value will be updated by the backend. - Constraints: > > > - Datatype: Same datatype as in[0] #### out[1] Gives the position for each bounding box as a 4-tuple, (x1,y1,x2,y2). Positions in this output correspond to the position of the same box in out[0]. - Mandatory: true - Data type: backend specific - Shape: [num\_boxes,4]. Max and current dimension value for *num\_boxes* may differ, and will be updated by the backend. - Constraints: > > > - Datatype: Same datatype as in[3] #### out[2] Gives the batch index of each bounding box. Positions in this output correspond to the position of the same box in out[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [num\_boxes]. Max and current dimension value for *num\_boxes* may differ, and will be updated by the backend. ## GetSparseIndices Gets the M specified indices from a sparse tensor. See the CreateSparse op definition for a description of K and partially sparse tensors and a description of the indices tensor. References: - QNN: [CreateSparse](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#createsparse) - QNN: [GetSparseValues](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#getsparsevalues) - QNN: [SparseToDense](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#sparsetodense) ### Inputs #### in[0] input - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Must be a sparse tensor. ### Parameters None ### Outputs #### out[0] indices - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32, QNN\_DATATYPE\_INT\_32 - Shape: [M, K] where 0 < K <= N. M may be dynamically sized and K is fixed. ## GetSparseValues Gets the M specified values from a sparse tensor. See the CreateSparse op definition for a description of K and partially sparse tensors and a description of the indices tensor. References: - QNN: [CreateSparse](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#createsparse) - QNN: [GetSparseIndices](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#getsparseindices) - QNN: [SparseToDense](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#sparsetodense) ### Inputs #### in[0] input - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N with shape \([D\_0,...,D\_{N-1}]\) - Constraints: > > > - Must be a sparse tensor. ### Parameters None ### Outputs #### out[0] values - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N-K+1 with shape \([M,D\_{K},...,D\_{N-1}]\) - Constraints: > > > - Datatype: Same datatype as in[0] ## GridSample Computes the output using input values and pixel locations from *grid*. For each output location the *grid* specifies input pixel locations x and y for 4D case and x, y, and z for 5D case, which are used to interpolate the output value. References: - ONNX: [GridSample](https://github.com/onnx/onnx/blob/main/docs/Operators.md#GridSample) - PyTorch: [GridSample](https://pytorch.org/docs/stable/generated/torch.nn.functional.grid_sample.html) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: 4D of shape: [batch, height, width, channel] or 5D of shape: [batch, depth, height, width, channel] - Constraints: > > > - Shape: Rank(in[0]) must equal 4 or 5 #### in[1] Grid : specifies the sampling pixel location normalized by the spatial dimensions of in[0]. Therefore, most values should be in the range of [-1, 1]. For example, values x = -1, y = -1 is the left-top pixel of in[0] and x = 1, y = 1 is the right-bottom pixel of in[0]. Note if values are outside the range [-1, 1] then the corresponding outputs will be handled using the method specified by *padding\_mode*. - Mandatory: true - Data type: backend specific - Shape: 4D of shape: [batch, height\_out, width\_out, 2] or 5D of shape: [batch, depth\_out, height\_out, width\_out, 3] - Constraints: > > > - Shape: Same rank as in[0] ### Parameters #### align\_corners If true, the maximum and minimum (1 and -1) are considered as referring to the center points of in[0] corner pixels. Otherwise, the maximum and minimum refer to the corner points of in[0] corner pixels. Note that pixels of in[0] are considered as squares rather than points. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### mode Determines the interpolation method. Supported values are 0: BILINEAR, 1: NEAREST. Note when inputs are 5D and *mode* is set to QNN\_OP\_GRID\_SAMPLE\_MODE\_BILINEAR the interpolation mode used internally will be trilinear. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - BILINEAR = 0, > - NEAREST = 1 #### padding\_mode Determines how the outputs are handled when *grid* has values outside the range [-1 ,1]. When padding\_mode == QNN\_OP\_GRID\_SAMPLE\_PADDING\_MODE\_ZEROS: use 0 for out-of-bound grid locations. When padding\_mode == QNN\_OP\_GRID\_SAMPLE\_PADDING\_MODE\_BORDER: use border values for out-of-bound grid locations. When padding\_mode == QNN\_OP\_GRID\_SAMPLE\_PADDING\_MODE\_REFLECTION: use values reflected by the border for out-of-bound locations. Note for locations far away from the border, it will keep reflecting until the location is in bounds of [-1, 1]. For example, given a pixel location x = -3.5 this will be reflected by the border at -1 and now becomes x’ = 1.5. Now this new pixel location will be reflected by the border at 1 and becomes x’’ = 0.5. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - ZEROS = 0, > - BORDER = 1, > - REFLECTION = 2 ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: 4D of shape: [batch, height\_out, width\_out, channel] or 5D of shape: [batch, depth\_out, height\_out, width\_out, channel] - Constraints: > > > - Shape: Same rank as in[0] ## GroupNorm Applies group normalization to the input tensor. The operation divides the channels into groups and computes within each group the mean (\(\mu\)) and variance (\(\sigma\)) for normalization. The values in the output tensor are computed as \[\mbox{out[0]}[b,h,w,c] = \frac{(\mbox{in[0]}[b,h,w,c] - \mu)}{\sqrt{\sigma^2 + \epsilon}} \* \gamma + \beta\] where gamma (\(\gamma\)), beta (\(\beta\)) and epsilon (\(\epsilon\)) are parameters. References: - ONNX: [GroupNormalization](https://github.com/onnx/onnx/blob/main/docs/Operators.md#GroupNormalization) - PyTorch: [torch.nn.GroupNorm](https://pytorch.org/docs/stable/generated/torch.nn.GroupNorm.html#torch.nn.GroupNorm) - TensorFlow: [tf.keras.layers.GroupNormalization](https://www.tensorflow.org/api_docs/python/tf/keras/layers/GroupNormalization) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N, note that the last dimension in the input is the channel i.e. [.., channel]. - Constraints: > > > - Shape: Rank > 0 > - Shape: channel must be evenly divisible by group #### in[1] gamma (\(\gamma\)) - Mandatory: false - Data type: backend specific - Shape: 1D of shape [channel] - Default: {1,..,1} #### in[2] beta (\(\beta\)) - Mandatory: false - Data type: backend specific - Shape: 1D of shape [channel] - Default: {0,..,0} ### Parameters #### epsilon (\(\epsilon\)) - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar #### group - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## Gru Performs a single Gated Recurrent Unit (GRU) layer. The GRU operation is described by the following equations: \[\begin{split}{\Large r\_{t}} &= {\Large \sigma(W\_{xr} x\_{t} + W\_{hr} h\_{t-1} + b\_{xr} + b\_{hr})} \\ \\ {\Large z\_{t}} &= {\Large \sigma(W\_{xz} x\_{t} + W\_{hz} h\_{t-1} + b\_{xz} + b\_{hz})} \\ \\ {\Large n\_{t}} &= \begin{cases} {\Large \phi(W\_{xn} x\_{t} + (r\_{t} \odot h\_{t-1}) W\_{hn} + b\_{xn} + b\_{hn})} & {\Large \text{if linear\_before\_reset = 0;}}, \\ \\ {\Large \phi(W\_{xn} x\_{t} + (r\_{t} \odot (W\_{hn} h\_{t-1} + b\_{hn})) + b\_{xn})} & {\Large \text{otherwise.}} \\ \end{cases} \\ \\ {\Large h\_{t}} &= {\Large (1 - z\_{t}) \odot n\_{t} + z\_{t} \odot h\_{t-1}} \\ \\\end{split}\] where \(x\_{t}\) is the input, \(r\_{t}\) is the reset gate, \(z\_{t}\) is the update gate, \(n\_{t}\) is the new gate, \(h\_{t}\) is the output state, \(\sigma\) is the logistic sigmoid function, \(\phi\) is the tanh activation function, \(\odot\) is the element-wise product of two vectors. \(W\_{xr}\) is the input-to-reset weight matrix, \(W\_{hr}\) is the recurrent-to-reset weight matrix, \(b\_{xr}\) is the input-reset gate bias, \(b\_{hr}\) is the recurrent-reset gate bias, \(W\_{xz}\) is the input-to-update weight matrix, \(W\_{hz}\) is the recurrent-to-update weight matrix, \(b\_{xz}\) is the input-update gate bias, \(b\_{hz}\) is the recurrent-update gate bias, \(W\_{xn}\) is the input-to-new weight matrix, \(W\_{hn}\) is the recurrent-to-new weight matrix, \(b\_{xn}\) is the input-new gate bias, \(b\_{hn}\) is the recurrent-new gate bias. The output state is maintained internal to the operation across multiple time-steps and inferences. This internal state can be reset at each inference by reading from in[13]. The decision whether or not to reset the internal state is made based on the value of the reset signal in[14]. If the value of this input is non-zero the internal state is reset from the output state input, or set to all zero values if the optional state input is not connected. References: - ONNX: [ops::GRU](https://github.com/onnx/onnx/blob/main/docs/Operators.md#GRU) - PyTorch: [GRU](https://pytorch.org/docs/stable/generated/torch.nn.GRU.html) ### Inputs #### in[0] X : The input \(x\_{t}\). - Mandatory: true - Data type: backend specific - Shape: 3D of shape [seq\_length, batch\_size, input\_size] if time\_major equals true or 3D of shape [batch\_size, seq\_length, input\_size] if time\_major equals false. #### in[1] input-to-update weights \(W\_{xz}\). - Mandatory: true - Data type: backend specific - Shape: [hidden\_size, input\_size] #### in[2] input-to-reset weights \(W\_{xr}\). - Mandatory: true - Data type: backend specific - Shape: [hidden\_size, input\_size] #### in[3] input-to-new weights \(W\_{xn}\). - Mandatory: true - Data type: backend specific - Shape: [hidden\_size, input\_size] #### in[4] recurrent-to-update weights \(W\_{hz}\). - Mandatory: true - Data type: backend specific - Shape: [hidden\_size, hidden\_size] #### in[5] recurrent-to-reset weights \(W\_{hr}\). - Mandatory: true - Data type: backend specific - Shape: [hidden\_size, hidden\_size] #### in[6] recurrent-to-new weights \(W\_{hn}\). - Mandatory: true - Data type: backend specific - Shape: [hidden\_size, hidden\_size] #### in[7] input-to-update gate bias \(b\_{xz}\). - Mandatory: false - Data type: backend specific - Shape: [hidden\_size] - Default: {0,..,0} #### in[8] input-to-reset gate bias \(b\_{xr}\). - Mandatory: false - Data type: backend specific - Shape: [hidden\_size] - Default: {0,..,0} #### in[9] input-to-new gate bias \(b\_{xn}\). - Mandatory: false - Data type: backend specific - Shape: [hidden\_size] - Default: {0,..,0} #### in[10] recurrent-to-update gate bias \(b\_{hz}\). - Mandatory: false - Data type: backend specific - Shape: [hidden\_size] - Default: {0,..,0} #### in[11] recurrent-to-reset gate bias \(b\_{hr}\). - Mandatory: false - Data type: backend specific - Shape: [hidden\_size] - Default: {0,..,0} #### in[12] recurrent-to-new gate bias \(b\_{hn}\). - Mandatory: false - Data type: backend specific - Shape: [hidden\_size] - Default: {0,..,0} #### in[13] initial\_h : Initial value of the output state (\(h\_{t}\)). When not specified it is assumed to be 0. - Mandatory: false - Data type: backend specific - Shape: [1, batch\_size, hidden\_size] - Default: {0,..,0} #### in[14] reset : Determines if the internal state should be reset. When set to true the internal state is reset by the input in[13] if it is provided, otherwise it is set to all zero values. Note that reset is used to indicate the reset of the internal state at the beginning of an inference pass across all batch elements at time-step 0. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: 0D containing scalar value - Default: 1 ### Parameters #### direction Specifies if the RNN is 0 : FORWARD or 1 : REVERSE. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - FORWARD = 0, > - REVERSE = 1 #### linear\_before\_reset During the computation of the output from the new gate, if *linear\_before\_reset* == 0 then the linear transformation is applied before multiplying by the output of the reset gate. Otherwise, the output from the new gate is multiplied by the output of the reset gate before applying the linear transformation. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 #### time\_major Determines the dimension order of the 3D main input and output. When time\_major is true, the 1st dimension of in[0] and out[0] corresponds to seq\_length dimension while the 2nd dimension is batch. When time\_major is false, the two dimensions are reversed. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 ### Outputs #### out[0] \(Y\) : A tensor that concatenates all the intermediate output values of the output state (\(h\_{t}\)). - Mandatory: true - Data type: backend specific - Shape: 3D of shape [seq\_length, batch\_size, hidden\_size] if time\_major equals true or 3D of shape [batch\_size, seq\_length, hidden\_size] if time\_major equals false. #### out[1] \(Y\_{h}\) : the final output state (\(h\_{t}\)) of the input sequence. - Mandatory: true - Data type: backend specific - Shape: [1, batch\_size, hidden\_size] ## HadamardTransform Performs Hadamard Transform operation on the last dimension of the input tensor. \[\mbox{out[0]} = \text{HadamardTransform}(\mbox{in[0]}) \* \mbox{scale}\] ### Inputs #### in[0] Input tensor to be transformed. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N ### Parameters #### scale Optional scaling factor applied to the output. This is a multiplicative factor applied after the transform. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1.0 ### Outputs #### out[0] Output tensor after applying the Hadamard Transform and scale. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## HardSwish The hard swish operation computes: out[0] = in[0] * max(0, min(6, (x + 3))) / 6 Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_HARD_SWISH](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ac88e58ac12cdd6ef3e20d7204ec6f33e) - TensorFlow: [hard_swish](https://github.com/tensorflow/models/blob/master/official/modeling/activations/swish.py) ### Inputs #### in[0] input activation. - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## HeatMapMaxKeyPoint Localize the maximum keypoints from heatmaps. This operation approximates the accurate maximum keypoint scores and indices after bicubic upscaling by using Taylor expansion up to the quadratic term. A bounding box is represented by its upper-left corner coordinate (x1,y1) and lower-right corner coordinate (x2,y2) in the original image. A valid bounding box should satisfy x1 <= x2 and y1 <= y2. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_HEATMAP_MAX_KEYPOINT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a5ffccf92d127766a741225ff7ad6f743) ### Inputs #### in[0] Tensor representing the heatmaps. - Mandatory: true - Data type: backend specific - Shape: 4D represented as [num\_boxes, heatmap\_height, heatmap\_width, num\_keypoints] #### in[1] Bounding boxes - Mandatory: true - Data type: backend specific - Shape: 2D represented as [num\_boxes, 4], each with format [x1, y1, x2, y2] representing bounding-box coordinates. ### Parameters None ### Outputs #### out[0] Keypoint scores - Mandatory: true - Data type: backend specific - Shape: [num\_boxes, num\_keypoints] - Constraints: > > > - Datatype: Same datatype as in[0] #### out[1] Keypoint locations - Mandatory: true - Data type: backend specific - Shape: [num\_boxes, num\_keypoints, 2], the second dimension organized as [keypoint\_x, keypoint\_y] - Constraints: > > > - Datatype: Same datatype as in[1] ## If If conditional operation. References: - ONNX: [If](https://github.com/onnx/onnx/blob/main/docs/Operators.md#if) - TensorFlow: [Cond](https://tensorflow.org/api_docs/python/tf/cond) - Caffe2: [If](https://caffe2.ai/docs/operators-catalogue.html#if) ### Inputs #### in[0] Condition indicating which branch to execute. - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: ### Parameters #### then\_graph Name of the subgraph to execute when the condition is true. - Mandatory: true - Data type: QNN\_DATATYPE\_STRING - Shape: scalar #### else\_graph Name of the subgraph to execute when the condition is false. - Mandatory: false - Data type: QNN\_DATATYPE\_STRING - Shape: scalar - Default: parameter not used unless set ### Outputs None ## Im2Col Extracts sliding kernel-sized blocks from a batched and multi-channeled input tensor and arranges them into batched columns, one for each iteration of the kernel. References: - PyTorch: [Unfold](https://pytorch.org/docs/stable/generated/torch.nn.Unfold.html) ### Inputs #### in[0] Input tensor - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel] ### Parameters #### kernel\_size The size of the sliding block over all channels of the input. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_kernel, width\_kernel] #### stride Defines stride for 2D spatial (i.e. height and width) axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Paddings along the height and width dimensions of the input tensor. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[pad\_top, pad\_bottom], [pad\_left, pad\_right]] #### dilation Dilation value along each spatial axis (i.e. height and width) of the input. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_dilation, width\_dilation] - Default: [1, 1] - Constraints: > > > - Value: Dilations must be > 0 ### Outputs #### out[0] Output tensor containing the rearranged data - Mandatory: true - Data type: backend specific - Shape: [batch, channel \* height\_kernel \* width\_kernel, L], where L is the number of columns made as a result of the sliding kernel. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: The value of L must be \(\prod\_i(\text{floor}(\frac{spatial\\_dimensions[i] + 2 \* padding[i]- dilation[i] \* (kernel\\_size[i] -1) - 1}{stride[i]} + 1))\), with spatial\_dimensions being the height and width of the input image ## ImageProjectionTransform Applies a projective transform to the image. This operator produces an output that is the result of transforming the input image based on the following definition, along with necessary interpolation: For a 3x3 transform matrix expressed as below: [a0, a1, a2, b0, b1, b2, c0, c1, 1 ] the operation maps the output point (x, y) to a transformed input point (x’, y’) expressed by (x’, y’) = ((a0 x + a1 y + a2) / k, (b0 x + b1 y + b2) / k), where k = c0 x + c1 y + 1 References: - TensorFlow: [Image Transform](https://www.tensorflow.org/addons/api_docs/python/tfa/image/transform) ### Inputs #### in[0] Input image. - Mandatory: true - Data type: backend specific - Shape: 4D of shape [batch,height,width,depth] #### in[1] Projective transform matrix. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: containing the non-unity elements of the 3x3 matrix expressed above. ### Parameters #### interpolation\_mode Determines the interpolation method. Supported values are 0: BILINEAR, 1: NEAREST\_NEIGHBOR. Default: BILINEAR - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - BILINEAR = 0, > - NEAREST\_NEIGHBOR = 1 ### Outputs #### out[0] Transformed image of the Same shape and size as input. - Mandatory: true - Data type: backend specific - Shape: 4D of shape [batch,height,width,depth] ## InstanceNorm Applies instance normalization to the input tensor. If mode is MU\_SIGMA, region ACROSS\_SPATIAL, and normalize\_variance is true, values in the output tensor are computed as \[\mbox{out[0]}[b,h,w,c] = \frac{(\mbox{in[0]}[b,h,w,c] - \mu\_{b,c})\*\gamma}{\sqrt{\sigma\_{b,c}^2 + \epsilon}} + \beta\] else if mode is MU\_SIGMA, region ACROSS\_SPATIAL, and normalize\_variance is false, values in the output tensor are computed as \[\mbox{out[0]}[b,h,w,c] = \mbox{in[0]}[b,h,w,c] - \mu\_{b,c}\] where gamma (\(\gamma\)), beta (\(\beta\)), and epsilon (\(\epsilon\)) are parameters and the mean (\(\mu\_{b,c}\)) is \[\mu\_{b,c} = \frac{\sum\_{h,w}\mbox{in[0]}[b,h,w,c]}{HW}\] where H and W are \[H = \mbox{shape}(\mbox{in[0]})[h]\] and \[W = \mbox{shape}(\mbox{in[0]})[w]\] and the variance (\(\sigma\_{b,c}^2\)) is \[\sigma\_{b,c}^2 = \frac{\sum\_{h,w}(\mbox{in[0]}[b,h,w,c] - \mu\_{b,c})^2}{HW} .\] If mode is RMS with region ACROSS\_ALL, values in the output tensor are computed as \[\mbox{out[0]}[b,h,w,c] = \frac{\mbox{in[0]}[b,h,w,c] \*\gamma}{\sqrt{\mbox{RMS}\_{b}}} + \beta\] where \(\mbox{RMS}\_{b}\) is \[\mbox{RMS}\_{b} = \sum\_{h,w,c}(\mbox{in[0]}[b,h,w,c])^2 + \epsilon.\] If mode is RMS with region ACROSS\_CHANNEL, values in the output tensor are computed as \[\mbox{out[0]}[b,h,w,c] = \frac{\mbox{in[0]}[b,h,w,c] \*\gamma}{\sqrt{\mbox{RMS}\_{b,h,w}}} + \beta\] where \(\mbox{RMS}\_{b,h,w}\) is \[\mbox{RMS}\_{b,h,w} = \sum\_{c}(\mbox{in[0]}[b,h,w,c])^2 + \epsilon.\] and gamma (\(\gamma\)), beta (\(\beta\)), and epsilon (\(\epsilon\)) are parameters References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_INSTANCE_NORMALIZATION](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a1e47d64cc1ac622e6fc515d602e1cd55) - QNN: [BroadcastingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: n-dimensional, note that the last dimension in the input is the channel, […, channel]. - Constraints: > > > - Shape: rank n > 2 #### in[1] gamma (\(\gamma\)). Is applied element-wise across the channel dimension of the mean-subtracted input activation in[0]. This op supports Unidirectional broadcasting from in[1] to in[0]. - Mandatory: true - Data type: backend specific - Shape: 1D of shape [channel] #### in[2] beta (\(\beta\)). Is applied element-wise across the channel dimension of the normalized input activation in[0]. This op supports Unidirectional broadcasting from in[2] to in[0]. - Mandatory: false - Data type: backend specific - Shape: 1D of shape [channel] - Default: {0,..,0} ### Parameters #### epsilon (\(\epsilon\)) - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1e-12 #### mode - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - MU\_SIGMA = 0, > - RMS = 1 #### normalize\_variance - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 #### region - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - ACROSS\_SPATIAL = 0, > - ACROSS\_CHANNEL = 1, > - ACROSS\_ALL = 2 ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: n-dimensional - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## IsInf Applies logical IsInf on the input elementwise. Returns true for infinity values detected, otherwise returns false. \[\begin{split}\mbox{out[0]} = \text{isInf}(\mbox{in[0]}) = \begin{cases} 1 & \text{if(}\ \mbox{in[0]} = \text{inf AND detect\_positive = 1)}, \\ 1 & \text{if(}\ \mbox{in[0]} = \text{-inf AND detect\_negative = 1)}, \\ 0 & \text{otherwise.} \end{cases}\end{split}\] References: - ONNX: [IsInf](https://github.com/onnx/onnx/blob/main/docs/Operators.md#IsInf) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16 - Shape: a tensor of rank N ### Parameters #### detect\_negative Determines if negative infinity is mapped to true. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 #### detect\_positive Determines if positive infinity is mapped to true. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: a tensor of rank N - Constraints: > > > - Shape: Same shape as in[0] ## IsNan Applies logical IsNan on the input elementwise. Returns true for NaN values detected, otherwise returns false. \[\mbox{out[0]} = \text{isnan}(\mbox{in[0]})\] References: - ONNX: [IsNaN](https://github.com/onnx/onnx/blob/main/docs/Operators.md#IsNaN) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, QNN\_DATATYPE\_FLOAT\_16 - Shape: a tensor of rank N ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: a tensor of rank N - Constraints: > > > - Shape: Same shape as in[0] ## L2Norm Applies an L2 normalization on a tensor along the specified axis. For a 1-D tensor with axis = 0, this computes \[\mbox{out[0]} = \frac{\mbox{in[0]}}{\mbox{max}(||\mbox{in[0]}||\_2,\epsilon)}\] For in[0] with more dimensions, this independently normalizes each 1-D slice along the dimension *axis*. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_L2_NORMALIZATION](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0abf295dee59560ff29d435226ec4c24bd) - TensorFlow: [L2 Normalize](https://www.tensorflow.org/api_docs/python/tf/math/l2_normalize) - ONNX: [LpNormalization](https://github.com/onnx/onnx/blob/master/docs/Operators.md#LpNormalization) ### Inputs #### in[0] Input activation - Mandatory: true - Data type: backend specific - Shape: N-dimensional - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axis Scalar index specifying dimension along which to normalize input. Both scalar *axis* and its tensor counterpart *axes* are optional, but at least one of them must be provided. However, if *axes* is non-empty, the value of *axis* is ignored. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Constraints: > > > - Value: Must be in range [0,N) #### axes Dimensions along which to normalize input. Each element must be in range [0,N) and must be unique. Both scalar *axis* and its tensor counterpart *axes* are optional, but at least one of them must be provided. When both are specified, values of *axes* take precedence over scalar *axis*. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Default: {0,..,0} - Constraints: > > > - Shape: 0 < M <=N #### epsilon (\(\epsilon\)) Positive valued, L2 norm lower bound. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1e-12 ### Outputs #### out[0] Normalized Output - Mandatory: true - Data type: backend specific - Shape: N-dimensional - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## L2Pool2d Performs a 2D L2-pooling on the input activation tensor. The values in the output tensor are computed as: \[\mbox{out[0]}[b,i,j,c] = \sqrt{\sum\_{di,dj}(\mbox{in[0]}[b, \mbox{stride}[0] \* i + di, \mbox{stride}[1] \* j + dj, c])^2}\] Pooling is performed over the 2D spatial shape of the input activation tensor, i.e. over it’s [height, width] sub-shape. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_L2_POOL_2D](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a2fb636e30d8853f9fa1a395e30660e92) - ONNX: [LpPool](https://github.com/onnx/onnx/blob/master/docs/Operators.md#LpPool) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel] ### Parameters #### filter\_size Defines the pool filter size for 2D spatial axes of in[0]. Number of elements to pool from = filter\_size[0] \* filter\_size[1] - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [filter\_height, filter\_width] #### stride Defines the pool stride size for 2D spatial axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 2D spatial axes in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] ### Outputs #### out[0] output activation The output 2D spatial dimensions are functions of the filter\_size, stride, and pad\_amount. shape(out[0])[height_out] = floor((pad_amount[0,0] + shape(in[0])[height] + pad_amount[0,1] - filter_size[0]) / stride[0] + 1) shape(out[0])[width_out] = floor((pad_amount[1,0] + shape(in[0])[width] + pad_amount[1,1] - filter_size[1]) / stride[1] + 1) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, height\_out, width\_out, channel] - Constraints: > > > - Datatype: Same datatype as in[0] ## LayerNorm Applies layer normalization to the input tensor along the specified *axes*. For a ND input tensor, with a 1D *axes* parameter of length M, the operation computes a ND mean tensor (\(\mu\)) and ND variance tensor (\(\sigma\)) with shapes *moments\_shape* whose *i-th* dimension is given by \[\mbox{shape}(\mbox{moments\_shape})[\mbox{i}] = 1\] if *i* is in *axes* and \[\mbox{shape}(\mbox{moments\_shape})[\mbox{i}] = \mbox{shape}(\mbox{in[0]})[\mbox{i}]\] otherwise. The values in the output tensor are computed as \[\mbox{out[0]}[b,h,w,c] = \frac{(\mbox{in[0]}[b,h,w,c] - \mu) \* \gamma}{\sqrt{\sigma^2 + \epsilon}} + \beta\] where gamma (\(\gamma\)), beta (\(\beta\)) and epsilon (\(\epsilon\)) are parameters. References: - TensorFlow: [tf.keras.layers.LayerNormalization](https://www.tensorflow.org/api_docs/python/tf/keras/layers/LayerNormalization) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 #### in[1] gamma (\(\gamma\)) Is applied element-wise across the normalization axes of the mean-subtracted input activation in[0]. This op supports Unidirectional broadcasting from in[1] to in[0]. The *i-th* dimension of gamma (\(\gamma\)) is given by \[\mbox{shape}(\gamma)[\mbox{i}] = \mbox{shape}(\mbox{in[0]})[\mbox{axes[i]}]\] - Mandatory: false - Data type: backend specific - Shape: a tensor of rank M, M <= size(axes) - Default: {1,..,1} #### in[2] beta (\(\beta\)) Is applied element-wise across the normalization axes of the normalized input activation in[0]. This op supports Unidirectional broadcasting from in[2] to in[0]. The *i-th* dimension of beta (\(\beta\)) is given by \[\mbox{shape}(\beta)[\mbox{i}] = \mbox{shape}(\mbox{in[0]})[\mbox{axes[i]}]\] - Mandatory: false - Data type: backend specific - Shape: a tensor of rank M, M <= size(axes) - Default: {0,..,0} ### Parameters #### epsilon (\(\epsilon\)) - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.001 #### axes A list of dimensions along which to normalize. Each value must be in range [0,N-1] and must be unique. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Constraints: > > > - Shape: 0 < M <= N ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## Logit Computes the logit (aka the inverse sigmoid) of the input element-wise. When *epsilon* (\(\epsilon\)) is provided the input is clamped to a range of [\(\epsilon\), 1 - \(\epsilon\)]. Otherwise, input is not clamped. Note the input domain of this function is (0, 1). Inputs outside this range may result in undefined behavior. \[\begin{split}\mbox{out[0]} &= \text{ln(}\frac{\text{z}}{\text{z - 1}}\text{)} \\ \text{z} &=\begin{cases} \mbox{in[0]} & \text{if(epsilon ($\epsilon$) not provided)}, \\ \text{max(min(}\mbox{in[0]}\text{, 1 - $\epsilon$), $\epsilon$)} & \text{Otherwise}. \\ \end{cases} \\\end{split}\] References: - PyTorch: [ops::Logit](https://docs.pytorch.org/docs/stable/special.html#torch.special.logit) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 #### in[1] epsilon (\(\epsilon\)) : Determines the input clamp bound [\(\epsilon\), 1 - \(\epsilon\)] - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: 1D of shape [1] - Default: None - Constraints: > > > - Value: Must be <= 0.5 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Same shape as in[0] > - Datatype: Same datatype as in[0] ## LogSoftmax Computes the log softmax of the input activations: out[0] = in[0] * beta - log(reduce_sum(exp(in * beta), axis)) Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_LOG_SOFTMAX](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a9e6c534786db67a647aeefcdafc3af0e) - TensorFlow: [ops::LogSoftmax](https://www.tensorflow.org/api_docs/python/tf/nn/log_softmax) - QNN: [ReduceSum](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#reducesum) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axis - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: must be in range [0, N-1] #### beta - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1.0 ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension - Constraints: > > > - Shape: Same shape as in[0] ## Lrn Performs Local Response Normalization on input data over the local regions defined below. A square sum is computed over a region of size *2R + 1*, where *R* is the radius. Let \[\begin{split}M\_1(x) &= \mbox{max}(0, x - R) \newline \\ M\_2(d,x) &= \mbox{min}(\mbox{shape}(\mbox{in[0]})[d], x + R). \newline \\\end{split}\] Note that *N* is the rank of in[0]. For across channel LRN, the square sum, *S*, is computed using this formula: \[S[n,...,c] = \sum\_{j=M\_1(c)}^{M\_2(N-1,c)}(\mbox{in[0]}[n,...,j])^2.\] For within channel LRN, the square sum, *S*, is computed over the spatial dimensions as \[S[n,h,w,c] = \sum\_{i=M\_1(h)}^{M\_2(1,h)}\sum\_{j=M\_1(w)}^{M\_2(2,w)}(\mbox{in[0]}[n,i,j,c])^2.\] The output is computed using \[\mbox{out}[0] = \frac{\mbox{in}[0]}{(B + \alpha S)^\beta}.\] where *B* is the bias. Note that some frameworks scale \(\alpha\) based upon a given size. The QNN definition does not scale \(\alpha\). References: - AlexNet: [AlexNet](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) - ONNX: [ops::LRN](https://github.com/onnx/onnx/blob/master/docs/Operators.md#LRN) - TensorFlow: [Local Response Normalization](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization) - Android NDK NeuralNetworks: [ANEURALNETWORKS_LOCAL_RESPONSE_NORMALIZATION](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a876ccb0f3e6555637c5e278a7715fc05) ### Inputs #### in[0] Input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 > - Shape: Rank N == 4 for within channel LRN ### Parameters #### alpha (\(\alpha\)) Scaling parameter - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1 #### beta (\(\beta\)) Exponent value - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.5 #### bias An offset usually positive to avoid division by zero - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1 #### radius Radius of the normalization window - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar #### region Set to QNN\_OP\_LRN\_REGION\_ACROSS\_CHANNEL to perform across channel LRN. Set to QNN\_OP\_LRN\_REGION\_WITHIN\_CHANNEL to perform within channel LRN. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - ACROSS\_CHANNEL = 0, > - WITHIN\_CHANNEL = 1 ### Outputs #### out[0] Normalized Output - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Same shape as in[0] ## Lstm Performs one or more time steps in a Long Short-Term Memory (LSTM) layer. The LSTM operation is described by the following equations: \[\begin{split}{\Large i\_{t}} &= {\Large \sigma\_(W\_{xi}x\_{t}+W\_{hi}h\_{t-1}+W\_{ci}C\_{t-1}+b\_i)} \\ \\ {\Large f\_{t}} &= {\Large \sigma\_(W\_{xf}x\_{t}+W\_{hf}h\_{t-1}+W\_{cf}C\_{t-1}+b\_f)} \\ \\ {\Large C\_{t}} &= {\Large clip(f\_{t} \odot C\_{t-1} + i\_{t} \odot \phi(W\_{xc}x\_{t}+W\_{hc}h\_{t-1}+b\_c),\ t\_{cell})} \\ \\ {\Large o\_{t}} &= {\Large \sigma\_(W\_{xo}x\_{t}+W\_{ho}h\_{t-1}+W\_{co}C\_{t}+b\_o)} \\ \\ {\Large h\_{t}} &= \begin{cases} {\Large clip(W\_{proj}(o\_t \odot \phi(C\_t))+b\_{proj},\ t\_{proj})} & {\Large \text{if there is a projection;}} \\ \\ {\Large o\_{t} \odot \phi(C\_{t})} & {\Large \text{otherwise.}} \\ \end{cases} \\ \\\end{split}\] where \(x\_{t}\) is the input, \(i\_{t}\) is the input gate, \(f\_{t}\) is the forget gate, \(C\_{t}\) is the cell state, \(o\_{t}\) is the output gate, \(h\_{t}\) is the output state, \(\sigma\) is the logistic sigmoid function, \(\phi\) is the tanh activation function, \(W\_{xi}\) is the input-to-input weight matrix, \(W\_{hi}\) is the recurrent-to-input weight matrix, \(W\_{ci}\) is the cell-to-input weight matrix, \(b\_i\) is the input gate bias, \(W\_{xf}\) is the input-to-forget weight matrix, \(W\_{hf}\) is the recurrent-to-forget weight matrix, \(W\_{cf}\) is the cell-to-forget weight matrix, \(b\_f\) is the forget gate bias, \(W\_{xc}\) is the input-to-cell weight matrix, \(W\_{hc}\) is the recurrent-to-cell weight matrix, \(b\_c\) is the cell bias, \(W\_{xo}\) is the input-to-output weight matrix, \(W\_{ho}\) is the recurrent-to-output weight matrix, \(W\_{co}\) is the cell-to-output weight matrix, \(b\_o\) is the output gate bias, \(W\_{proj}\) is the projection weight matrix, \(b\_{proj}\) is the projection bias, \(t\_{cell}\) is the threshold for clipping the cell state, and \(t\_{proj}\) is the threshold for clipping the projected output. \(\odot\) is the element-wise product of two vectors. The operation can be stateful or stateless depending on the dimensionality of the \(x\_{t}\) input. If the input is 2D, the operation is stateless and a single time-step is executed. The initial output state and cell state are read from in[10] and in[11] respectively. If the input is 3D, the 2nd dimension represents the number of time-steps to be executed if time\_major is false. If time\_major is true, then the 1st dimension of the input represents the time-steps and the 2nd dimension is the batch dimension. The output state and cell state are maintained internal to the operation across multiple time-steps and inferences. These internal states can be reset at each inference by reading from in[10] and in[11] respectively. The decision whether or not to reset the internal state is made based on the value of the reset signal in[24]. If the value of this input is non-zero the internal state is reset from the state inputs, or set to all zero values if the optional state inputs are not connected. The operation has the following independently optional inputs: - The cell-to-input weights (\(W\_{ci}\)), cell-to-forget weights (\(W\_{cf}\)) and cell-to-output weights (\(W\_{co}\)) either all have values or neither of them have values (i.e., all set to null). If they have values, the peephole optimization is used. - The input-to-input weights (\(W\_{xi}\)), recurrent-to-input weights (\(W\_{hi}\)) and input gate bias (\(b\_i\)) either all have values, or none of them have values. If they have no values, coupling of input and forget gates (CIFG) is used, in which case the input gate (\(i\_{t}\)) is calculated using the following equation instead: \[{\Large i\_{t} = 1 - f\_{t}}\] In case peephole optimization is used and CIFG is not used cell-to-input (\(W\_{ci}\)) weights must be present. Otherwise, the cell-to-input weights must have no value. - The projection weights (\(W\_{proj}\)) is required only for the recurrent projection layer, and should otherwise have no value. - The projection bias (\(b\_{proj}\)) may (but not required to) have a value if the recurrent projection layer exists, and should otherwise have no value. - The four layer normalization weights either all have values or none of them have values. Additionally, if CIFG is used, input layer normalization weights tensor is omitted and the other layer normalization weights either all have values or none of them have values. Layer normalization is used when the values of all the layer normalization weights are present. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_LSTM](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ad0377e8c305e596fb7f64ff896671fc5) ### Inputs #### in[0] The input \(x\_{t}\). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [batch\_size, input\_size] or 3D of shape [batch\_size, time\_steps, input\_size] if time\_major equals false or 3D of shape [time\_steps, batch\_size, input\_size] if time\_major equals true. - Constraints: > > > - Shape: Rank(in[0]) must equal 2 or 3 #### in[1] input-to-forget weights \(W\_{xf}\). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [num\_units, input\_size] #### in[2] input-to-cell weights \(W\_{xc}\). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [num\_units, input\_size] #### in[3] input-to-output weights \(W\_{xo}\). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [num\_units, input\_size] #### in[4] recurrent-to-forget weights \(W\_{hf}\). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [num\_units, output\_size] #### in[5] recurrent-to-cell weights \(W\_{hc}\). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [num\_units, output\_size] #### in[6] recurrent-to-output weights \(W\_{ho}\). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [num\_units, output\_size] #### in[7] forget gate bias \(b\_f\). - Mandatory: true - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[8] cell bias \(b\_c\). - Mandatory: true - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[9] output gate bias \(b\_o\). - Mandatory: true - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[10] output state (in) \(h\_{t-1}\). - Mandatory: false - Data type: backend specific - Shape: 2D of shape [batch\_size, output\_size] - Default: {0,..,0} #### in[11] cell state (in) \(C\_{t-1}\). - Mandatory: false - Data type: backend specific - Shape: 2D of shape [batch\_size, num\_units] - Default: {0,..,0} #### in[12] The input layer normalization weights. Used to rescale normalized inputs to activation at input gate. - Mandatory: false - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[13] The forget layer normalization weights. Used to rescale normalized inputs to activation at forget gate. - Mandatory: false - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[14] The cell layer normalization weights. Used to rescale normalized inputs to activation at cell gate. - Mandatory: false - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[15] The output layer normalization weights. Used to rescale normalized inputs to activation at output gate. - Mandatory: false - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[16] input-to-input weights \(W\_{xi}\). - Mandatory: false - Data type: backend specific - Shape: 2D of shape [num\_units, input\_size], where “num\_units” corresponds to the number of cell units. #### in[17] recurrent-to-input weights \(W\_{hi}\). - Mandatory: false - Data type: backend specific - Shape: 2D of shape [num\_units, output\_size] where “output\_size” corresponds to either the number of cell units (i.e., “num\_units”), or the first dimension of the “projection\_weights”, if defined. #### in[18] cell-to-input weights \(W\_{ci}\). It is a diagonal matrix by definition, and is expressed as a 1D vector. - Mandatory: false - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[19] cell-to-forget weights \(W\_{cf}\). It is a diagonal matrix by definition, and is expressed as a 1D vector. - Mandatory: false - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[20] cell-to-output weights \(W\_{co}\). It is a diagonal matrix by definition, and is expressed as a 1D vector. - Mandatory: false - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[21] input gate bias \(b\_i\). - Mandatory: false - Data type: backend specific - Shape: 1D of shape [num\_units] #### in[22] projection weights \(W\_{proj}\). - Mandatory: false - Data type: backend specific - Shape: 2D of shape [output\_size, num\_units] #### in[23] projection bias \(b\_{proj}\). - Mandatory: false - Data type: backend specific - Shape: 1D of shape [output\_size] #### in[24] reset : Determines if the internal state should be reset. When set to true the internal states are reset by the inputs in[10] and in[11] if they are provided, otherwise they are set to all zero values. Note that reset is only applicable to a 3D input \(x\_{t}\) and used to indicate the reset of the internal states at the beginning of an inference pass across all batch elements at time-step 0. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: 0D containing scalar value - Default: 1 ### Parameters #### direction The ‘direction’ of computation for the LSTM op. Used to achieve the functionality of Bi-directional LSTM by using a combination of individual LSTM ops configured in opposite directions. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - FORWARD = 0, > - REVERSE = 1 #### cell\_clip\_threshold The clipping threshold (\(t\_{cell}\)) for the cell state, such that values are bound within [-cell\_clip, cell\_clip]. If set to 0.0 clipping is disabled. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### output\_clip\_threshold The clipping threshold (\(t\_{proj}\)) for the output from the projection layer, such that values are bound within [-proj\_clip, proj\_clip]. If set to 0.0 clipping is disabled. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### time\_major Determines the dimension order of the 3D main input and output. When equal to true, the 1st dimension of in[0] and out[0] corresponds to time\_step dimension while the 2nd dimension is batch. When time\_major is false, the two dimensions are reversed. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 #### input\_gate\_qscale The quantization scale of the intermediate result of matrix multiplication, i.e input to layer normalization at input gate \(i\_{t}\) realized by the following expression: \((W\_{xi}x\_{t} + W\_{hi}h\_{t-1} + W\_{ci}C\_{t-1})\) - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### forget\_gate\_qscale The quantization scale of the intermediate result of matrix multiplication, i.e input to layer normalization at forget gate \(f\_{t}\) realized by the following expression: \((W\_{xf}x\_{t} + W\_{hf}h\_{t-1} + W\_{cf}C\_{t-1})\) - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### cell\_gate\_qscale The quantization scale of the intermediate result of matrix multiplication, i.e input to layer normalization at cell gate \(C\_{t}\) realized by the following expression: \((W\_{xc}x\_{t} + W\_{hc}h\_{t-1})\) - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### output\_gate\_qscale The quantization scale of the intermediate result of matrix multiplication, i.e input to layer normalization at output gate \(o\_{t}\) realized by the following expression: \((W\_{xo}x\_{t} + W\_{ho}h\_{t-1} + W\_{co}C\_{t})\) - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### hidden\_state\_offset The quantization offset of the hidden state \(h\_{t}\), i.e. input to projection realized by the following expression: \((o\_t \odot \phi(C\_t))\) - Mandatory: false - Data type: backend specific - Shape: scalar - Default: 0.0 #### hidden\_state\_qscale The quantization scale of the hidden state \(h\_{t}\), i.e. input to projection realized by the following expression: \((o\_t \odot \phi(C\_t))\) - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 ### Outputs #### out[0] output state (out) (\(h\_{t}\)). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [batch\_size, output\_size] or 3D of shape [batch\_size, time\_steps, output\_size] if time\_major equals false, or 3D of shape [time\_steps, batch\_size, output\_size] if time\_major equals true. - Constraints: > > > - Shape: Same Rank as in[0] #### out[1] cell state (out) (\(C\_{t}\)). - Mandatory: true - Data type: backend specific - Shape: 2D of shape [batch\_size, num\_units] #### out[2] output (\(o\_{t}\)) : This is the the current “output state (out)” value. If out[0] is 2D, it is identical to out[0]. If out[0] is 3D, it contains the values of the final time-step. - Mandatory: true - Data type: backend specific - Shape: 2D of shape [batch\_size, output\_size]. #### out[3] input\_gate - Mandatory: false - Data type: backend specific - Shape: 2D of shape [batch\_size, num\_units]. #### out[4] forget\_gate - Mandatory: false - Data type: backend specific - Shape: 2D of shape [batch\_size, num\_units]. #### out[5] cell\_gate - Mandatory: false - Data type: backend specific - Shape: 2D of shape [batch\_size, num\_units]. #### out[6] output\_gate - Mandatory: false - Data type: backend specific - Shape: 2D of shape [batch\_size, output\_size]. #### out[7] hidden\_state - Mandatory: false - Data type: backend specific - Shape: 2D of shape [batch\_size, output\_size]. ## MaskedSoftmax Applies a Softmax operation on masked portions of the input tensor. For each *batch* the mask tensor is broadcast on the input before softmax computation. A mask tensor must be provided in either an *UNCOMPRESSED* or *COMPRESSED* format depending on the *mode* selected. See in[1] for details on how a boolean mask can be converted to an *UNCOMPRESSED* or *COMPRESSED* mask tensor. ### Inputs #### in[0] Input data - Mandatory: true - Data type: backend specific - Shape: 4D of shape [batch, height, width, channel] - Constraints: > > > - Shape: When mode is set to COMPRESSED width == channel. #### in[1] Mask tensor: The representation of this 2D tensor is determined by the *mode* selected. When *mode* is set to *UNCOMPRESSED* M = channel or set to *COMPRESSED* M = number of sequences. Consider a boolean mask where a mask value of 1 indicates the dimension on which Softmax should be performed and a mask value of 0 indicates the dimension that Softmax will not be performed. An uncompressed mask can be made from a boolean mask tensor by adding -1 or subtracting by 1 element-wise and multiplying the intermediate result element-wise by a large value. mask = [[1,1,1,0,1]] uncompressed_mask = (mask .+ -1) .* 10000 // uncompressed_mask = [[0,0,0,-10000,0]] Copy to clipboard A compressed mask can be made from multiple boolean mask tensors of vector lengths that are concatenated into a single batch and summed across the 2nd axis. For Example: Let there be 3 mask tensors that correspond to sequences of inputs that were used to make in[0] where 0’s represent where padding was added to make them the max sequence length. mask1 = [1,0,0,0] mask2 = [1,1,1,0] mask3 = [1,1,1,1] Copy to clipboard The concatenated mask would then be the following: concatenated_mask = [ [1,0,0,0], [1,1,1,0], [1,1,1,1]] Copy to clipboard The compressed mask representation would be made from summing across the 2nd axis: compressed_mask = [[1,3,4]] Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: 2D of shape [batch, M] - Constraints: > > > - Value: When mode is set to COMPRESSED the sum of values in each batch must be <= channel. ### Parameters #### mode Determines the MaskedSoftmax Operation that is performed. See in[1] for details on mask tensor format for each of the modes. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - UNCOMPRESSED = 0, > - COMPRESSED = 1 ### Outputs #### out[0] Output Activation - Mandatory: true - Data type: backend specific - Shape: 4D of shape [batch, height, width, channel] - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## MatMul Performs a batched matrix multiplication on the innermost two dimensions of the operands in[0] and in[1]. Batch dimensions of in[0] and in[1] will be broadcast when possible. To be broadcast, the smaller dimension must be equal to 1, or omitted if the ranks of the input tensors differ. Batch dimensions of out[0] must match the batch dimensions of in[0] and in[1] after being broadcast. For example, if in[0] is of shape [10, 256, 128] and in[1] is of shape [5, 1, 128, 64], the input batch dimensions are [10] and [5, 1] respectively, and will be broadcast to [5, 10]. out[0] will be of shape [5, 10, 256, 64]. Let \[A = \mbox{in}[0], B = \mbox{in}[1], C = \mbox{out}[0], b = \mbox{in}[2].\] When \(\mbox{transpose\_in0} = 0\) and \(\mbox{transpose\_in1} = 0\), the operation satisfies \[C[..., i, j] = \sum^{m1}\_{k=0}(A[..., i, k] \* B[..., k, j] + b[j]), \forall i \in [0,m0], \forall j \in [0,n1].\] When \(\mbox{transpose\_in0} = 0\) and \(\mbox{transpose\_in1} = 1\), the operation satisfies \[C[..., i, j] = \sum^{m1}\_{k=0}(A[..., i, k] \* B[..., j, k] + b[j]), \forall i \in [0,m0], \forall j \in [0,m1].\] When \(\mbox{transpose\_in0} = 1\) and \(\mbox{transpose\_in1} = 0\), the operation satisfies \[C[..., i, j] = \sum^{n1}\_{k=0}(A[..., k, i] \* B[..., k, j] + b[j]), \forall i \in [0,n0], \forall j \in [0,n1].\] When \(\mbox{transpose\_in0} = 1\) and \(\mbox{transpose\_in1} = 1\), the operation satisfies \[C[..., i, j] = \sum^{n1}\_{k=0}(A[..., k, i] \* B[..., j, k] + b[j]), \forall i \in [0,n0], \forall j \in [0,m1].\] Refer to MatMul backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - TensorFlow: [BatchMatMulV2](https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/batch-mat-mul-v2) ### Inputs #### in[0] Matrix operand: A - Mandatory: true - Data type: backend specific - Shape: […, m0, n0] of rank N >= 2 - Dynamic Shape: All dimensions can be dynamic. #### in[1] Matrix operand: B When transpose_in0 = 0 and transpose_in1 = 0: m1 = n0 When transpose_in0 = 0 and transpose_in1 = 1: n1 = n0 When transpose_in0 = 1 and transpose_in1 = 0: m1 = m0 When transpose_in0 = 1 and transpose_in1 = 1: n1 = m0 Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: […, m1, n1] of rank N >= 2 - Dynamic Shape: All dimensions can be dynamic. #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [n2] - Dynamic Shape: n2 can be dynamic. - Default: {0,..,0} ### Parameters #### transpose\_in0 - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### transpose\_in1 - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Matrix product: C When transpose_in0 = 0 and transpose_in1 = 0: m2 = m0 and n2 = n1 When transpose_in0 = 0 and transpose_in1 = 1: m2 = m0 and n2 = m1 When transpose_in0 = 1 and transpose_in1 = 0: m2 = n0 and n2 = n1 When transpose_in0 = 1 and transpose_in1 = 1: m2 = n0 and n2 = m1 Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: […, m2, n2] of rank N >= 2 - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Dynamic Shape: For each batch dimension, if the corresponding dimension for either *in[0]* or *in[1]* is dynamic, it too must be dynamic. If *transpose\_in0* = 0 and m0 is dynamic or if *transpose\_in0* = 1 and n0 is dynamic, m2 must be dynamic. If *transpose\_in1* = 0 and n1 is dynamic or if *transpose\_in1* = 1 and m1 is dynamic, n2 must be dynamic. ## Moments Calculates the mean and variance of an input tensor. See Moments backend definition for supported datatypes and constraints per backend References: - TensorFlow: [ops::Moments](https://www.tensorflow.org/api_docs/python/tf/nn/moments) ### Inputs #### in[0] Input activation - Mandatory: true - Data type: backend specific - Shape: ND, of length <= rank(in[0]) - Constraints: > > > - Shape: length <= rank(in[0]) > - Shape: Rank > 0 ### Parameters #### axes Axes along which to compute mean and variance - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: 1D. For batch normalization: Tensor of 1 element (batch only), or 3 elements for filters with shape [batch, height, width, depth] (e.g. pass axes=[0, 1, 2]). #### keep\_dims Produce moments with the same dimensionality as the input. Set to either QNN\_OP\_MOMENTS\_KEEP\_DIM true or false. - Mandatory: true - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar ### Outputs #### out[0] Mean - Mandatory: true - Data type: backend specific - Shape: If keepdims = QNN\_OP\_MOMENTS\_KEEP\_DIMS true, specified axes dims of in[0] become 1, others are same as in[0]. For keepdims = QNN\_OP\_MOMENTS\_KEEP\_DIMS false, specified axes of in[0] are omitted and others are as in[0]. - Constraints: > > > - Shape: Rank > 0 #### out[1] Variance - Mandatory: true - Data type: backend specific - Shape: If keepdims = QNN\_OP\_MOMENTS\_KEEP\_DIMS true, specified axes dims of in[0] become 1, others are same as in[0]. For keepdims = QNN\_OP\_MOMENTS\_KEEP\_DIMS false, specified axes of in[0] are omitted and others are as in[0] - Constraints: > > > - Shape: Rank > 0 ## MultiClassNms Filter bounding boxes across multiple classes in descending order of score using Non maximum suppression. Boxes that have a high IOU overlap with previously selected boxes are pruned. Additionally, bounding boxes with scores less than a specific threshold are pruned. The op also performs gather operation on any additional features corresponding to the filtered boxes. References: - TensorFlow: [ops::NonMaxSuppressionV5](https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/non-max-suppression-v5) ### Inputs #### in[0] Bounding boxes. Elements can be understood as 4-tuples of bounding box coordinates given in the form (y1,x1,y2,x2). - Mandatory: true - Data type: backend specific - Shape: [batch, num\_boxes, 4] #### in[1] Bounding box scores. The element at position [*batch*, *box*, *class*] is the score corresponding to *class* for the bounding box at the position [*batch*, *box*] in in[0]. - Mandatory: true - Data type: backend specific - Shape: [batch, num\_boxes, num\_classes] #### in[2..m] Additional feature vectors where m >=2. These features are also ‘filtered’ by performing ‘gather’ operations along the same indices as the filtered boxes. This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: false - Data type: backend specific - Shape: N-dimensional. dim[0] should match *batch*; dim[1] should match num\_boxes. - Constraints: > > > - Number: m >= 2 > - Shape: Rank > 0 ### Parameters #### iou\_threshold IoU threshold for the NMS algorithm. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar #### score\_threshold Boxes with scores lower than the threshold are filtered by the NMS algorithm. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### soft\_nms\_sigma Sigma parameter for Soft NMS. When this param is set to a non-zero value, boxes reduce the score of other overlapping boxes instead of directly causing them to be pruned. When the value of the param is 0, the NMS algorithm defaults to the hard version. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 ### Outputs #### out[0] Selected Output boxes. Each element can be understood as a 4-tuple with the same meaning as in[0]. - Mandatory: true - Data type: backend specific - Shape: [batch, max\_num\_boxes, 4]. max\_num\_boxes is expressed by maxDimensions in Qnn\_Tensor\_t. The number of valid boxes is specified by out[3]. - Constraints: > > > - Datatype: Same datatype as in[0] #### out[1] Selected Output box scores. Gives the score for the box in the corresponding position in out[0]. - Mandatory: true - Data type: backend specific - Shape: [batch, max\_num\_boxes]. max\_num\_boxes is expressed by maxDimensions in Qnn\_Tensor\_t. The number of valid scores is specified by out[3]. - Constraints: > > > - Datatype: Same datatype as in[1] #### out[2] Selected Output classes. Gives the class label for the box in the corresponding position in out[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [batch, max\_num\_boxes]. max\_num\_boxes is expressed by maxDimensions in Qnn\_Tensor\_t. The number of valid classes is specified by out[3]. #### out[3] Number of valid boxes per batch element that remain after NMS. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [batch] - Constraints: > > > - Value: <= max\_num\_boxes #### out[4..M] Selected features post NMS. They are filtered along axis 0 that matches the dimensions of the boxes. max\_num\_boxes is expressed by maxDimensions in Qnn\_Tensor\_t. The number of valid classes is specified by out[3]. The number of these output features must match the number of input features in in[2..m]. This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: false - Data type: backend specific - Shape: N-dimensional. dim[0] should match *batch*; dim[1] should match max\_num\_boxes. - Constraints: > > > - Datatype: Same datatypes as in[2..m] > - Number: M >= 4 > - Shape: Rank > 0 > - Value: (M - 3) must be equal to (m - 1) ## NonMaxSuppression Filters out boxes that have high intersection-over-union (IOU) overlap with previously selected boxes. Bounding boxes with a score less than *score\_threshold* are removed. Note when the number of valid boxes detected is less than *max\_selected\_indices*, out[0] is padded with the indices of the detected box containing the lowest score. If no valid boxes are detected then the output is padded with the indices to the first box. References: - ONNX: [NonMaxSuppression](https://github.com/onnx/onnx/blob/main/docs/Operators.md#NonMaxSuppression) ### Inputs #### in[0] Bounding boxes: Elements can be understood as 4-tuples of bounding box coordinates given in the form (y1,x1,y2,x2), where (y1, x1) and (y2, x2) represent a diagonal pair of corners. - Mandatory: true - Data type: backend specific - Shape: [batch, num\_boxes, 4] #### in[1] Bounding box scores. - Mandatory: true - Data type: backend specific - Shape: [batch, num\_classes, num\_boxes] ### Parameters #### iou\_threshold Represents the threshold used by NMS algorithm to determine whether boxes overlap too much. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Constraints: > > > - Value: Must be in range [0, 1] #### score\_threshold Boxes with scores lower than the threshold are filtered out by the NMS algorithm. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 #### max\_boxes\_selected Maximum number of boxes that can be selected per batch per class. Note that default 0 means there is no valid output boxes. Note that if value provided is greater than *num\_boxes* it is set to the value of num\_boxes. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 ### Outputs #### out[0] selected\_indices: Indices of the elements that have been kept by NMS algorithm. Each element can be understood as a 3-tuple with index format (batch\_index, class\_index, box\_index). - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [max\_selected\_indices, 3]. max\_selected\_indices = batch \* num\_classes \* max\_boxes\_selected, where the valid number of selected indices is specified by out[1]. #### out[1] Valid number of selected indices per batch after NMS. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [batch] - Constraints: > > > - Value: Must be <= max\_selected\_indices ## NonZero Generates a 2D output tensor containing the indices to nonzero elements of in[0]. Note this operator provides indices of all detected nonzero elements in the initial portion of the output tensor. The remaining elements of the output tensor are populated with a fill value of -1. References: - ONNX: [ops::NonZero](https://github.com/onnx/onnx/blob/main/docs/Operators.md#NonZero) - PyTorch: [NonZero](https://pytorch.org/docs/stable/generated/torch.nonzero.html) - TensorFlow: [Where](https://www.tensorflow.org/api_docs/python/tf/where) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] Indices to nonzero elements of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32 - Shape: [M, N], where M is the total number of elements of in[0]. ## Nv12ToRgb Transform Nv12 to RGB (or BRG). ### Inputs #### in[0] Input tensor. - Mandatory: true - Data type: backend specific - Shape: [b, w\*h\*3/2] where w and h are the width and height of the image. ### Parameters #### output\_order Controls the order of the output tensor. Set to QNN\_OP\_NV12\_TO\_RGB\_OUTPUT\_ORDER\_RGB the output will be in RGB. Set to QNN\_OP\_NV12\_TO\_RGB\_OUTPUT\_ORDER\_BGR, the output will be BGR. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Values: > > > - RGB = 0, > - BGR = 1 ### Outputs #### out[0] Output tensor. - Mandatory: true - Data type: backend specific - Shape: [b,h,w,3] where w and h are the width and height of the image. ## Nv21ToRgb Transform Nv21 to RGB (or BRG). ### Inputs #### in[0] Input tensor. - Mandatory: true - Data type: backend specific - Shape: [b, w\*h\*3/2] where w and h are the width and height of the image. ### Parameters #### output\_order Controls the order of the output tensor. Set to QNN\_OP\_NV21\_TO\_RGB\_OUTPUT\_ORDER\_RGB the output will be in RGB. Set to QNN\_OP\_NV21\_TO\_RGB\_OUTPUT\_ORDER\_BGR, the output will be BGR. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Values: > > > - RGB = 0, > - BGR = 1 ### Outputs #### out[0] Output tensor. - Mandatory: true - Data type: backend specific - Shape: [b,h,w,3] where w and h are the width and height of the image. ## OneHot Creates a one-hot encoded tensor. Locations in indices will take on QNN\_OP\_ONE\_HOT\_PARAM\_ON\_VALUE while all other locations take on QNN\_OP\_ONE\_HOT\_PARAM\_OFF\_VALUE. Depth of one-hot locations can be specified with QNN\_OP\_ONE\_HOT\_PARAM\_DEPTH. References: - TensorFlow: [one_hot](https://www.tensorflow.org/api_docs/python/tf/one_hot) - ONNX: [OneHot](https://github.com/onnx/onnx/blob/master/docs/Operators.md#OneHot) ### Inputs #### in[0] Indices of one hot encoding - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32, QNN\_DATATYPE\_INT\_32 - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 > - Value: Index values must be in range [0, QNN\_OP\_ONE\_HOT\_PARAM\_DEPTH-1] ### Parameters #### depth Depth of the one-hot dimension. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar #### axis The axis to fill the one-hot dimension. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: N - Constraints: > > > - Value: in range [0, N] #### on\_value The value to fill at the indices provided by in[0]. - Mandatory: false - Data type: backend specific - Shape: scalar - Default: 1 #### off\_value The value to fill at all other indices not provided in in[0]. - Mandatory: false - Data type: backend specific - Shape: scalar - Default: 0 - Constraints: > > > - Datatype: Same datatype as QNN\_OP\_ONE\_HOT\_PARAM\_ON\_VALUE ### Outputs #### out[0] Output one-hot encoded tensor. - Mandatory: true - Data type: backend specific - Shape: Shape is the same as in[0], but with rank(out[0]) = N + 1. A dimension of QNN\_OP\_ONE\_HOT\_PARAM\_DEPTH inserted at the axis specified by QNN\_OP\_ONE\_HOT\_PARAM\_AXIS. - Constraints: > > > - Datatype: Same datatype as QNN\_OP\_ONE\_HOT\_PARAM\_ON\_VALUE ## Pack Packs the list of same rank tensors into a tensor with rank one higher than each tensor by packing them along the axis dimension. References: - TensorFlow: [Stack](https://www.tensorflow.org/api_docs/python/tf/stack) ### Inputs #### in[0..m] input tensors. m >=1 This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: true - Data type: backend specific - Shape: N-dimensional. Must be the same for all inputs. - Constraints: > > > - Number: m >= 1 > - Shape: Rank > 0 ### Parameters #### axis Dimension along which packing will happen - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Value <= (rank(in[0])) ### Outputs #### out[0] The packed output tensor - Mandatory: true - Data type: backend specific - Shape: - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: rank(out[0]) = rank(in[0]) + 1 ## Pad Pads input tensor with the appropriate value based on the scheme picked. Pad amount is a tensor with shape [N, 2] where N is the rank of the input. Pad amount [i, 0] identifies pad size to add before on dimension i and [i, 1] identifies pad size to add after on dimension i. The output tensor on that dimension will have a size equal to the input tensor plus the padding before and after. shape(out[0])[i] = pad_amount[i, 0] + shape(in[0])[i] + pad_amount[i, 1] Copy to clipboard The pad value is applicable when scheme used is CONSTANT. Client is responsible to provide a value that is appropriate to the operation consuming the padded input. Pad value will be ignored for other schemes. Mirror padding scheme uses tensor data to fill the pad amount. The padded region does not use the border value when using MIRROR\_REFLECT whereas it uses the border value when using MIRROR\_SYMMETRIC. For MIRROR\_REFLECT, the before and after pad amounts must not be greater than shape(in[0])[i] - 1. For MIRROR\_SYMMETRIC, the before and after pad amounts must not be greater than shape(in[0])[i]. Edge padding scheme applies padding with the edge values of the tensor in each dimension. Refer Pad backend definition for supported data type and layouts for each backend. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_PAD_V2](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0afe2c4f9e541dcc278509921fd2c8a502) - TensorFlow: [Mirror Pad](https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/mirror-pad) - ONNX: [ops::Pad](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Pad) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters #### scheme - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Values: > > > - CONSTANT = 0, > - MIRROR\_SYMMETRIC = 1, > - MIRROR\_REFLECT = 2, > - EDGE = 3 #### pad\_amount - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [rank(in[0]), 2] #### pad\_constant\_value - Mandatory: false - Data type: backend specific - Shape: scalar - Default: 0 ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same rank as in[0] ## PoolAvg2d Performs 2D average pooling on the input activation tensor by averaging a subset of the input tensor values according to the filter\_size and stride, effectively downsampling the input data into the output activation tensor. Average pooling is performed over 2D spatial shape of the input activation tensor, i.e. over it’s [height, width] sub-shape. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_AVERAGE_POOL_2D](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a12e6b53aadbd3736c38f1a159adea788) - ONNX: [AveragePool](https://github.com/onnx/onnx/blob/master/docs/Operators.md#AveragePool) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel] ### Parameters #### filter\_size Defines filter size for 2D spatial axes of in[0]. Number of elements to average = filter\_size[0]\*filter\_size[1] - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [filter\_height, filter\_width] #### stride Defines stride for 2D spatial axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 2D spatial axes of in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### count\_pad\_for\_edges Include pad elements when calculating average for the edges. 0 = do not include pad. Any other value will include padding into average calculation. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### rounding\_mode Indicate the rounding mode used in truncating output dimensions to integer values. Available options: 0: FLOOR, 1: CEIL. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - FLOOR = 0, > - CEIL = 1 ### Outputs #### out[0] output activation The output 2D spatial dimensions are functions of the filter\_size, stride, pad\_amount and rounding\_mode. shape(out[0])[height_out] = ROUND((pad_amount[0,0] + shape(in[0])[height] + pad_amount[0,1] - filter_size[0]) / stride[0] + 1) shape(out[0])[width_out] = ROUND((pad_amount[1,0] + shape(in[0])[width] + pad_amount[1,1] - filter_size[1]) / stride[1] + 1) where ROUND = floor() or ceil(), based on rounding_mode. Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, height\_out, width\_out, channel] - Constraints: > > > - Datatype: Same datatype as in[0] ## PoolAvg3d Performs 3D average pooling on the input activation tensor by averaging a subset of the input tensor values according to the filter\_size and stride, effectively downsampling the input data into the output activation tensor. Average pooling is performed over 3D spatial shape of the input activation tensor, i.e. over it’s [depth, height, width] sub-shape. References: - ONNX: [AveragePool](https://github.com/onnx/onnx/blob/master/docs/Operators.md#AveragePool) - TensorFlow: [AveragePool3D](https://www.tensorflow.org/api_docs/python/tf/nn/avg_pool3d) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, depth, height, width, channel] ### Parameters #### filter\_size Defines filter size for 3D spatial axes of in[0]. Number of elements to average = filter\_size[0] \* filter\_size[1] \* filter\_size[2] - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [filter\_depth, filter\_height, filter\_width] #### stride Defines stride for 3D spatial axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [depth\_stride, height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 3D spatial axes of in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3,2] with format [[depth\_pad\_before, depth\_pad\_after], [height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### count\_pad\_for\_edges Include pad elements when calculating average for the edges. 0 = do not include pad. Any other value will include padding into average calculation. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### rounding\_mode Indicate the rounding mode used in truncating output dimensions to integer values. Available options: 0: FLOOR, 1: CEIL. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - FLOOR = 0, > - CEIL = 1 ### Outputs #### out[0] output activation : The output 3D spatial dimensions are functions of the filter\_size, stride, pad\_amount and rounding\_mode. depth_out = ROUND((pad_amount[0,0] + Shape(in[0])[depth] + pad_amount[0,1] - filter_size[0]) / stride[0] + 1) height_out = ROUND((pad_amount[1,0] + Shape(in[0])[height] + pad_amount[1,1] - filter_size[1]) / stride[1] + 1) width_out = ROUND((pad_amount[2,0] + Shape(in[0])[width] + pad_amount[2,1] - filter_size[2]) / stride[2] + 1) where ROUND = floor() or ceil(), based on *rounding_mode*. Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, depth\_out, height\_out, width\_out, channel] - Constraints: > > > - Datatype: Same datatype as in[0] ## PoolMax2d Performs 2D maximum pooling on the input activation tensor by computing maximum value in a subset of the input tensor values according to the filter\_size and stride, effectively downsampling the input data into the output activation tensor. Maximum pooling is performed over 2D spatial shape of the input activation tensor, i.e. over it’s [height, width] sub-shape. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_MAX_POOL_2D](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a0f227a4d98ad5af31f7fd4d255d246ce) - ONNX: [MaxPool](https://github.com/onnx/onnx/blob/master/docs/Operators.md#MaxPool) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel] ### Parameters #### filter\_size Defines max pool filter size for 2D spatial axes of in[0]. Number of elements to pool from = filter\_size[0] \* filter\_size[1] - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [filter\_height, filter\_width] #### stride Defines max pool stride size for 2D spatial axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 2D spatial axes of in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### rounding\_mode Indicate the rounding mode used in truncating output dimensions to integer values. Available options: 0: FLOOR, 1: CEIL. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - FLOOR = 0, > - CEIL = 1 ### Outputs #### out[0] output activation The output 2D spatial dimensions are functions of the filter\_size, stride, pad\_amount and rounding\_mode. shape(out[0])[height_out] = ROUND((pad_amount[0,0] + shape(in[0])[height] + pad_amount[0,1] - filter_size[0]) / stride[0] + 1) shape(out[0])[width_out] = ROUND((pad_amount[1,0] + shape(in[0])[width] + pad_amount[1,1] - filter_size[1]) / stride[1] + 1) where ROUND = floor() or ceil(), based on rounding_mode. Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, height\_out, width\_out, channel] - Constraints: > > > - Datatype: Same datatype as in[0] ## PoolMax3d Performs 3D maximum pooling on the input activation tensor by computing maximum value in a subset of the input tensor values according to the filter\_size and stride, effectively downsampling the input data into the output activation tensor. Maximum pooling is performed over 3D spatial shape of the input activation tensor, i.e. over it’s [depth, height, width] sub-shape. References: - ONNX: [MaxPool](https://github.com/onnx/onnx/blob/master/docs/Operators.md#MaxPool) - TensorFlow: [MaxPool3D](https://www.tensorflow.org/api_docs/python/tf/nn/max_pool3d) - PyTorch: [MaxPool3D](https://pytorch.org/docs/stable/generated/torch.nn.MaxPool3d.html) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, depth, height, width, channel] ### Parameters #### filter\_size Defines max pool filter size for 3D spatial axes of in[0]. Number of elements to pool from = filter\_size[0] \* filter\_size[1] \* filter\_size[2] - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [filter\_depth, filter\_height, filter\_width] #### stride Defines max pool stride size for 3D spatial axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [depth\_stride, height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 3D spatial axes of in[0]. Pad value = 0. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3,2] with format [[depth\_pad\_before, depth\_pad\_after], [height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### rounding\_mode Indicate the rounding mode used in truncating output dimensions to integer values. Available options: 0: FLOOR, 1: CEIL. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - FLOOR = 0, > - CEIL = 1 ### Outputs #### out[0] output activation The output 3D spatial dimensions are functions of the filter\_size, stride, pad\_amount, and rounding\_mode. shape(out[0])[depth_out] = ROUND((pad_amount[0,0] + shape(in[0])[depth] + pad_amount[0,1] - filter_size[0]) / stride[0] + 1) shape(out[0])[height_out] = ROUND((pad_amount[1,0] + shape(in[0])[height] + pad_amount[1,1] - filter_size[1]) / stride[1] + 1) shape(out[0])[width_out] = ROUND((pad_amount[2,0] + shape(in[0])[width] + pad_amount[2,1] - filter_size[2]) / stride[2] + 1) where ROUND = floor() or ceil(), based on rounding_mode. Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, depth\_out, height\_out, width\_out, channel] - Constraints: > > > - Datatype: Same datatype as in[0] ## Prelu The **P**arametric **re**ctified **l**inear **u**nit operation computes: out[0] = in[1] * in[0] for in[0] < 0 out[0] = in[0] for in[0] >= 0 Copy to clipboard The coefficient tensor in[1] is applied element-wise to input activation tensor in[0]. Only unidirectional broadcasting is possible, from in[1] to in[0]. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_PRELU](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a82e7e7c709ad117a0178203217be30cf) - ONNX: [ops::PRELU](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Prelu) - QNN: [BroadcastingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 #### in[1] coefficients - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M, where 0 < M <= N - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] ## Quantize Implements: output = round(input / scale) + offset Copy to clipboard where *output* is limited to *out[0]*’s data type maximum and minimum. Note that *scale* and *offset* are determined from out[0]. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_QUANTIZE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a60e0015c8f08ed26d59afe92f728068d) - QNN: [Dequantize](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#dequantize) ### Inputs #### in[0] - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32, backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] - Mandatory: true - Data type: QNN\_DATATYPE\_SFIXED\_POINT\_4, QNN\_DATATYPE\_UFIXED\_POINT\_4, QNN\_DATATYPE\_SFIXED\_POINT\_8, QNN\_DATATYPE\_UFIXED\_POINT\_8, QNN\_DATATYPE\_SFIXED\_POINT\_16, QNN\_DATATYPE\_UFIXED\_POINT\_16, QNN\_DATATYPE\_SFIXED\_POINT\_32, QNN\_DATATYPE\_UFIXED\_POINT\_32, backend specific - Shape: Any - Constraints: > > > - Shape: Same shape as in[0] ## RandomUniformLike Generate a tensor with random values drawn from a uniform distribution. If the output tensor has dynamic dimensions, the shape of the output tensor is determined by the input shape tensor. The parameters of the uniform distribution are specified by low and high. Note random values drawn from uniform distribution may vary across backends. Refer to RandomUniformLike backend definition per backend support for dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - ONNX: [ops::RandomUniformLike](https://github.com/onnx/onnx/blob/main/docs/Operators.md#RandomUniformLike) - PyTorch: [torch.rand_like](https://docs.pytorch.org/docs/stable/generated/torch.rand_like.html) - TensorFlow: [tf.random.uniform](https://www.tensorflow.org/api_docs/python/tf/random/uniform) ### Inputs #### in[0] Input tensor : a 1D tensor specifying the shape of the expected output tensor. The value of *in[0]* is not used unless *out[0]* is of dynamic shape. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [N] - Constraints: > > > - If *out[0]* has dynamic dimensions, *in[0]* must be present. Otherwise, *in[0]* is ignored. > - Shape: Rank > 0 #### in[1] Seed to the random generator, if not specified will auto generate one. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: 1D of shape [1] - Default: None ### Parameters #### low Lower boundary of the output values. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 0.0 - Constraints: > > > - Value: Must be less than high #### high Upper boundary of the output values. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1.0 - Constraints: > > > - Value: Must be greater than low ### Outputs #### out[0] Output tensor with shape specified by in[0] and random values drawn from a uniform distribution. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N ## ReduceMax Reduces a tensor by computing the maximum of elements along given dimensions as specified in *axes*. If *keep\_dims* is true, the reduced dimensions are retained with length 1. Otherwise, the rank of the tensor is reduced by 1 for each entry in *axes*. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_REDUCE_MAX](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a257a84d4cfa9f6eef0997ddd591e9584) - ONNX: [ops::ReduceMax](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ReduceMax) ### Inputs #### in[0] Input activations - Mandatory: true - Data type: backend specific - Shape: N-dimensional - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axes A list of dimensions along which to reduce. Each value must be in range [0,N-1] and must be listed only once. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Constraints: > > > - Shape: 0 < M <= N #### keep\_dims If true, the resulting tensor has the same number of dimensions as the input tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output activations - Mandatory: true - Data type: backend specific - Shape: K-dimensional, where K = N if *keep\_dims* is true and K = max(1, N - M) otherwise K-dimensional, where K = N if *keep\_dims* is true and K = max(1, N - M) otherwise - Constraints: > > > - Datatype: Same datatype as in[0] ## ReduceMean Reduces a tensor by computing the mean of elements along given dimensions as specified in *axes*. If *keep\_dims* is true, the reduced dimensions are retained with length 1. Otherwise, the rank of the tensor is reduced by 1 for each entry in *axes*. Refer to ReduceMean backend definition for support of dynamic dimensions for each backend. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_MEAN](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a047fe95a35b27f45c05432b6ca18eb6c) - ONNX: [ops::ReduceMean](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ReduceMean) ### Inputs #### in[0] Input activations - Mandatory: true - Data type: backend specific - Shape: N-dimensional - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axes A list of dimensions along which to reduce. Each value must be in range [0,N-1] and must be listed only once. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Constraints: > > > - Shape: 0 < M <= N #### keep\_dims If true, the resulting tensor has the same number of dimensions as the input tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output activations - Mandatory: true - Data type: backend specific - Shape: K-dimensional, where K = N if *keep\_dims* is true and K = max(1, N - M) otherwise. - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each non reduced dimension, if shape(in[0])[i] is dynamic then the corresponding dimension in out[0] must be dynamic. If keep\_dims is set to true, the reduced dimensions are retained as static dimensions of length 1. ## ReduceMin Reduces a tensor by computing the minimum of elements along given dimensions as specified in *axes*. If *keep\_dims* is true, the reduced dimensions are retained with length 1. Otherwise, the rank of the tensor is reduced by 1 for each entry in *axes*. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_REDUCE_MIN](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0ab5934984ee3e89ffb703b0cac7d6a2f8) - ONNX: [ops::ReduceMin](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ReduceMin) ### Inputs #### in[0] Input activations - Mandatory: true - Data type: backend specific - Shape: N-dimensional - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axes A list of dimensions along which to reduce. Each value must be in range [0,N-1] and must be listed only once. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Constraints: > > > - Shape: 0 < M <= N #### keep\_dims If true, the resulting tensor has the same number of dimensions as the input tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output activations - Mandatory: true - Data type: backend specific - Shape: K-dimensional, where K = N if *keep\_dims* is true and K = max(1, N - M) otherwise - Constraints: > > > - Datatype: Same datatype as in[0] ## ReduceProd Reduces a tensor by computing the product of elements along given dimensions as specified in *axes*. If *keep\_dims* is true, the reduced dimensions are retained with length 1. Otherwise, the rank of the tensor is reduced by 1 for each entry in *axes*. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_REDUCE_PROD](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0aea51a05dfc337082df51b4a263eb0c5c) - ONNX: [ops::ReduceProd](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ReduceProd) ### Inputs #### in[0] Input activations - Mandatory: true - Data type: backend specific - Shape: N-dimensional - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axes A list of dimensions along which to reduce. Each value must be in range [0,N-1] and must be listed only once. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Constraints: > > > - Shape: 0 < M <= N #### keep\_dims If true, the resulting tensor has the same number of dimensions as the input tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output activations - Mandatory: true - Data type: backend specific - Shape: K-dimensional, where K = N if *keep\_dims* is true and K = max(1, N - M) otherwise - Constraints: > > > - Datatype: Same datatype as in[0] ## ReduceSum Reduces a tensor by computing the sum of elements along given dimensions as specified in *axes*. If *keep\_dims* is true, the reduced dimensions are retained with length 1. Otherwise, the rank of the tensor is reduced by 1 for each entry in *axes*. Refer to ReduceSum backend definition for support of dynamic dimensions for each backend. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_REDUCE_SUM](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0aa2b7674dd559dd3233523688ad71cb3e) - ONNX: [ops::ReduceSum](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ReduceSum) ### Inputs #### in[0] Input activations - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axes A list of dimensions along which to reduce. Each value must be in range [0,N-1] and must be listed only once. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Constraints: > > > - Shape: 0 < M <= N #### keep\_dims If true, the resulting tensor has the same number of dimensions as the input tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output activations - Mandatory: true - Data type: backend specific - Shape: a tensor of rank K, where K = N if *keep\_dims* is true and K = max(1, N - M) otherwise - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each non-reduced dimension, if shape(in[0])[i] is dynamic then the corresponding dimension in out[0] must be dynamic. If keep\_dims is set to true, the reduced dimensions are retained as static dimensions of length 1. ## ReduceSumSquare Reduces a tensor by computing the sum square of elements along given dimensions as specified in *axes*. If *keep\_dims* is true, the reduced dimensions are retained with length 1. Otherwise, the rank of the tensor is reduced by 1 for each entry in *axes*. If rank(*in[0]*) = 0, no reduction occurs and the output is simply *in[0]* squared. References: - ONNX: [ops::ReduceSumSquare](https://github.com/onnx/onnx/blob/main/docs/Operators.md#ReduceSumSquare) ### Inputs #### in[0] Input activations - Mandatory: true - Data type: backend specific - Shape: N-dimensional ### Parameters #### axes A list of dimensions along which to reduce. Each value must be in range [0,N-1] and must be listed only once. If rank(*in[0]*) = 0, this param is ignored. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Constraints: > > > - Shape: 0 < M <= N unless rank(*in[0]*) = 0 #### keep\_dims If true, the resulting tensor has the same number of dimensions as the input tensor. If rank(*in[0]*) = 0, this param is ignored. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output activations - Mandatory: true - Data type: backend specific - Shape: K-dimensional, where K = N if *keep\_dims* is true and K = max(0, N - M) otherwise, where N = rank(*in[0]*) and M = size(*axes*) - Constraints: > > > - Datatype: Same datatype as in[0] ## Relu The **Re**ctified **l**inear **u**nit operation computes: output = max(0, input). Copy to clipboard References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_RELU](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0abb2f979866b131c5089ba0caaecee656) - TensorFlow: [ops::Relu](https://www.tensorflow.org/api_docs/python/tf/nn/relu) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] > - Must have same data format as in[0] (e.g. both sparse or both dense) ## Relu1 The **Re**ctified **l**inear 1 **u**nit operation computes: output = min(1.f, max(-1.f, input)). Copy to clipboard **DEPRECATED:** Use [ReluMinMax](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#reluminmax) instead. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_RELU1](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a73b9a2ded1dda2925d2e73aec44d2e2e) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## Relu6 The **Re**ctified **l**inear 6 **u**nit operation computes: output = min(6, max(0, input)). Copy to clipboard **DEPRECATED:** Use [ReluMinMax](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#reluminmax) instead. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_RELU6](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a04a24c2d6f0aac4c3f5324c1d7764714) - TensorFlow: [ops::Relu6](https://www.tensorflow.org/api_docs/python/tf/nn/relu6) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## ReluMinMax The Rectified Linear Unit Min Max operation computes: output = min(max_value, max(min_value, input)), where min_value <= max_value Copy to clipboard The ReluMinMax rectifies values within min\_value and max\_value. It can support different types of Relu operations. For example, for Relu6, use min\_value=0 and max\_value=6. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_RELU6](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a04a24c2d6f0aac4c3f5324c1d7764714) - TensorFlow: [ops::Relu6](https://www.tensorflow.org/api_docs/python/tf/nn/relu6) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters #### min\_value The minimum value in Relu operation - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar #### max\_value The maximum value in Relu operation - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## Reshape Reshapes a tensor of N dimension to an output shape of M dimension while retaining the values of the input tensor. The number of elements implied by the output shape must be the same as the shape of the input tensor. Note that N can equal M. Refer to Reshape backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_RESHAPE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a535e7e99383ee49456c8671843b93a59) - ONNX: [ops::Reshape](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Reshape) - PyTorch: [torch.reshape](https://pytorch.org/docs/stable/generated/torch.reshape.html) - TensorFlow: [tf.reshape](https://www.tensorflow.org/api_docs/python/tf/reshape) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension - Dynamic Shape: All dimensions can be dynamic. #### in[1] Determines the shape of *out[0]*. Note that 0 and -1 are valid values where a 0 is interpreted as the corresponding dimension in *in[0]* and a -1, which may only occur up to one time if *in[0]* is dynamic, is a wildcard dimension that will be inferred from the remaining dimensions and the number of elements in *in[0]*. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: 1D of shape [M] - Constraints: > > > - If *out[0]* has dynamic dimensions, *in[1]* must be present. Otherwise, *in[1]* is ignored. > - Value: All values of 0 have to have an index < rank(in[0]). > - Value: May contain a single -1 value if *in[0]* has dynamic dimensions. > - Value: Values must be >= -1. > - Value: The specified shape must be compatible with shape(*in[0]*) in keeping the number of elements the same. ### Parameters None ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: a tensor of M dimension - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Shape(out[0])[i] == Shape(in[0])[i] if in[1][i] == 0 > - Dynamic Shape: Must have dynamic dimensions if *in[0]* has dynamic dimensions. ## Resize Resizes the spatial dimensions of an input tensor with shape [batch, D1, D2, …, Dn, channel], where D1, …, Dn are the spatial dimensions. Every value of the output tensor is calculated as a weighted average of sampling locations in the input tensor. References: - ONNX: [Resize](https://github.com/onnx/onnx/blob/main/docs/Operators.md#resize) ### Inputs #### in[0] Input image - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension with shape [batch, D1, D2, … Dn, channel], where D1, …, Dn are the spatial dimensions. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### exclude\_outside If true, the weight of sampling locations outside the tensor will be set to 0 and the weight will be renormalized so that their sum is 1.0. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### transformation\_mode Determines how to transform the coordinates in the original input tensor to the coordinates in the resized tensor. Note that the coordinates of each dimension are transformed individually. Supported values are 0: HALF\_PIXEL, 1: PYTORCH\_HALF\_PIXEL, 2: ALIGN\_CORNERS, 3: ASYMMETRIC. When transformation\_mode = HALF\_PIXEL: \[x\_{out} = (x\_{in} + 0.5) \* \mbox{scale} - 0.5\] When transformation\_mode = PYTORCH\_HALF\_PIXEL: \[\begin{split}x\_{out} &= (x\_{in} + 0.5) \* \mbox{scale} - 0.5\ (\text{if}\ \text{shape} (\mbox{out[0]})[axis\_{x}] > 1;) \\ x\_{out} &= 0\ (\text{otherwise}) \\\end{split}\] When transformation\_mode = ALIGN\_CORNERS: \[x\_{out} = \frac{x\_{in} \* (\text{shape}(\mbox{out[0]})[axis\_{x}] - 1)}{\text{shape} (\mbox{in[0]})[axis\_{x}] - 1}\] When transformation\_mode = ASYMMETRIC: \[x\_{out} = x\_{in} \* \mbox{scale}\] where \(x\_{in}\) is a coordinate of \(\text{shape} (\mbox{in[0]})[axis\_{x}]\), \(x\_{out}\) is a coordinate of \(\text{shape} (\mbox{out[0]})[axis\_{x}]\), \(\mbox{scale} = \frac{\text{shape} (\mbox{out[0]})[axis\_{x}]}{\text{shape} (\mbox{in[0]})[axis\_{x}]}\). - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - HALF\_PIXEL = 0, > - PYTORCH\_HALF\_PIXEL = 1, > - ALIGN\_CORNERS = 2, > - ASYMMETRIC = 3 #### interpolation\_mode Determines the interpolation method. Supported values are 0: NEAREST, 1: LINEAR, 2: CUBIC. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - NEAREST = 0, > - LINEAR = 1, > - CUBIC = 2 #### nearest\_mode Determines the rounding method used when *interpolation\_mode* is set to NEAREST. Supported values are 0: ROUND\_PREFER\_FLOOR, 1: ROUND\_PREFER\_CEIL, 2: FLOOR, 3 : CEIL. For the following example let *x* represent a value in the range (0, 1]. When QNN_OP_RESIZE_PARAM_NEAREST_MODE is set to QNN_OP_RESIZE_NEAREST_MODE_ROUND_PREFER_FLOOR : floor(x) (if x <= 0.5) ceil(x) otherwise. When QNN_OP_RESIZE_PARAM_NEAREST_MODE is set to QNN_OP_RESIZE_NEAREST_MODE_ROUND_PREFER_CEIL : ceil(x) (if x >= 0.5) floor(x) otherwise. Copy to clipboard - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - ROUND\_PREFER\_FLOOR = 0, > - ROUND\_PREFER\_CEIL = 1, > - FLOOR = 2, > - CEIL = 3 - Constraints: > > > - interpolation\_mode must be set to NEAREST for this parameter to be valid. #### cubic\_coeff Coefficient used in cubic interpolation. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: -0.75 - Constraints: > > > - interpolation\_mode must be set to CUBIC for this parameter to be valid. ### Outputs #### out[0] Output image - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension after resizing. - Constraints: > > > - Datatype: Same datatype as in[0] ## ResizeBilinear Resize a 4D image in the height and width dimensions, computing new pixel values by bilinear interpolation. Image will be distorted if the aspect ratio of the output does not match the aspect ratio of the input. The output height and width are defined by the shape of the output tensor, and not computed. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_RESIZE_BILINEAR](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a42bd92518e273b6716ecd56b571fcd3e) - TensorFlow: [ResizeMethod.BILINEAR](https://www.tensorflow.org/api_docs/python/tf/compat/v1/image/resize) ### Inputs #### in[0] Input image - Mandatory: true - Data type: backend specific - Shape: [batch,height,width,depth] ### Parameters #### align\_corners If true, the centers of the 4 corner pixels of the input and output tensors are aligned, preserving the values at the corner pixels - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 - Constraints: > > > - Value: If True half\_pixel\_centers parameter MUST be false #### half\_pixel\_centers True or False where value 0 is False and any other is True. If true, the pixels are assumed to be at (0.5,0.5). - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 - Constraints: > > > - Value: If True align\_corners parameter MUST be false #### antialias Determines if an antialiasing filter is used when downsampling. If set to true uses resampling filter by a factor of max(1, 1/scale) during downsampling so more input pixels contribute to an output pixel. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output image - Mandatory: true - Data type: backend specific - Shape: [batch,new\_height,new\_width,depth] - Constraints: > > > - Datatype: Same datatype as in[0] ## ResizeNearestNeighbor Resize a 4D image in the height and width dimensions, computing new pixel values by nearest-neighbor sampling. Image will be distorted if the aspect ratio of the output does not match the aspect ratio of the input. The output height and width are defined by the shape of the output tensor, and not computed. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_RESIZE_NEAREST_NEIGHBOR](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a633997ea5224dc4063d35b9f2b2ab84b) - TensorFlow: [ResizeMethod.NEAREST_NEIGHBOR](https://www.tensorflow.org/api_docs/python/tf/compat/v1/image/resize) ### Inputs #### in[0] Input image - Mandatory: true - Data type: backend specific - Shape: [batch,height,width,depth] ### Parameters #### align\_corners If true, the centers of the 4 corner pixels of the input and output tensors are aligned, preserving the values at the corner pixels. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 - Constraints: > > > - Value: If True half\_pixel\_centers parameter MUST be false #### half\_pixel\_centers True or False where value 0 is False and any other is True. If true, the pixels are assumed to be at (0.5,0.5). - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 - Constraints: > > > - Value: If True align\_corners parameter MUST be false ### Outputs #### out[0] Output image - Mandatory: true - Data type: backend specific - Shape: [batch,new\_height,new\_width,depth] - Constraints: > > > - Datatype: Same datatype as in[0] ## RmsNorm Applies Root Mean Square normalization to the input tensor across the dimensions determined by *axes*. Focuses on re-scaling invariance and normalizing the summed inputs. Values in the output tensor are computed as \[\mbox{out[0]}[b,h,w,c] = \frac{\mbox{in[0]}[b,h,w,c] \* \gamma}{\sqrt{\frac{1}{n}\sum\_{axes}^n(\mbox{in[0]}[b,h,w,c])^2 + \epsilon}} + \beta\] where n is the number of elements summed across the dimensions provided by axes, gamma (\(\gamma\)) and beta (\(\beta\)) are inputs, and epsilon (\(\epsilon\)) is a parameter. Refer to RmsNorm backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - QNN: [BroadcastingRules](https://docs.qualcomm.com/doc/80-63442-50/topic/terminology.html) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N, note that the last dimension in the input is the channel, [.., channel]. - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 #### in[1] gamma (\(\gamma\)) used for scaling normalized values. Is applied element-wise across the channel dimension of the input activation in[0]. This op supports Unidirectional broadcasting from in[1] to in[0]. The *i-th* dimension of gamma (\(\gamma\)) is given by \[\mbox{shape}(\gamma)[\mbox{i}] = \mbox{shape}(\mbox{in[0]})[\mbox{axes[i]}]\] - Mandatory: false - Data type: backend specific - Shape: a tensor of rank M, where M = size(axes) - Dynamic Shape: All dimensions can be dynamic. - Default: {1,..,1} - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[0])[axes[i]] is dynamic then shape(in[1])[i] must be dynamic or must compatible for broadcasting. #### in[2] beta (\(\beta\)) used for re-centering invariance, which is disabled by default. Is applied element-wise across the channel dimension of the normalized input activation in[0]. This op supports Unidirectional broadcasting from in[2] to in[0]. The *i-th* dimension of beta (\(\beta\)) is given by \[\mbox{shape}(\beta)[\mbox{i}] = \mbox{shape}(\mbox{in[0]})[\mbox{axes[i]}]\] - Mandatory: false - Data type: backend specific - Shape: a tensor of rank M, where M = size(axes) - Dynamic Shape: All dimensions can be dynamic. - Default: {0,..,0} - Constraints: > > > - Dynamic Shape: For each dimension, if shape(in[0])[axes[i]] is dynamic then shape(in[2])[i] must be dynamic or must compatible for broadcasting. ### Parameters #### epsilon (\(\epsilon\)) (\(\epsilon\)) is used for mathematical stability to prevent division by 0. - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1e-06 #### axes Determines the dimensions that Root Mean Squared Normalization is applied. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] - Constraints: > > > - Shape: 0 < M <=N > - Value: Must be in range [0,N-1] > - Value: Must be unique ### Outputs #### out[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic, then shape(out[0])[i] must be dynamic. ## RoiAlign Extract rectangular **R**egions **o**f **I**nterest from a feature map, and scale them to a uniform size by average pooling of bilinearly interpolated sample points within the region. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_ROI_ALIGN](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a2848b39dd4bfba78f2438fda0d9397a4) ### Inputs #### in[0] Input feature map - Mandatory: true - Data type: backend specific - Shape: [batch,height,width,num\_features] #### in[1] RoIs : Elements may be interpreted as 4-tuples of (x1,y1,x2,y2) giving the upper-left and bottom-right corners of the RoI. - Mandatory: true - Data type: backend specific - Shape: [num\_rois,4] #### in[2] Batch index in the feature map to which each RoI corresponds. Positions in this input correspond to the same box as the same position in in[1]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32, backend specific - Shape: [num\_rois] ### Parameters #### img\_size\_ratio The ratio between the original image and the input feature map, in the form [height\_ratio, width\_ratio]. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: [2] with elements [height\_ratio, width\_ratio] #### num\_samples\_y The number of interpolated sample points to use in the height dimension for each RoI. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Default: -1 #### num\_samples\_x The number of interpolated sample points to use in the width dimension for each RoI. - Mandatory: false - Data type: QNN\_DATATYPE\_INT\_32 - Shape: scalar - Default: -1 #### aligned If true, shift the box coordinates by -0.5 for a better alignment with the neighboring pixels. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 #### allow\_invalid\_roi When set to true invalid RoIs of in[1] are allowed. An invalid RoI is defined as having coordinate values where (x2 - x1 = 0) and (y2 - y1 = 0). Note that the corresponding feature map in out[0] is set to all zeros for the invalid RoI. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output feature map - Mandatory: true - Data type: backend specific - Shape: [num\_rois,out\_height,out\_width,num\_features]. Max and current values may differ for *num\_rois*, and current value will be updated by the backend. - Constraints: > > > - Datatype: Same datatype as in[0] ## RoiPooling Extract rectangular **R**egions **o**f **I**nterest from a feature map, and scale them to a uniform size by max pooling of bilinearly interpolated sample points within the region. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_ROI_POOLING](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a6736198af337b2efbdb0b6b64dee7fe4) ### Inputs #### in[0] Input feature map - Mandatory: true - Data type: backend specific - Shape: [batch,height,width,num\_features] #### in[1] RoIs. Elements may be interpreted as 4-tuples of (x1,y1,x2,y2) giving the upper-left and bottom-right corners of the RoI. - Mandatory: true - Data type: backend specific - Shape: [num\_rois,4] #### in[2] Batch index in the feature map to which each RoI corresponds. Positions in this input correspond to the same box as the same position in in[1]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [num\_rois] ### Parameters #### img\_size\_ratio The ratio between the original image and the input feature map, in the form [height\_ratio, width\_ratio]. - Mandatory: true - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: [2] with elements [height\_ratio, width\_ratio] ### Outputs #### out[0] Output feature map - Mandatory: true - Data type: backend specific - Shape: [num\_rois,out\_height,out\_width,num\_features]. Max and current values may differ for *num\_rois*, and current value will be updated by the backend. - Constraints: > > > - Datatype: Same datatype as in[0] ## ScatterElements Takes three input tensors: *data*, *indices*, and *updates* of the same rank N and optional parameter *axis* to create an output tensor. Out[0] is produced by creating a copy of in[0] and then updating it with values that are specified by in[2] at the index positions specified by in[1]. Note that for each entry in *updates* the target index of out[0] is the same index as the entry itself with the exception of the index-value in the *axis* dimension which is obtained from *indices*. Example for a 3-D tensor case is performed as below with *reduction* set to QNN\_OP\_SCATTER\_ELEMENTS\_REDUCTION\_NONE: output[indices[i][j][k]][j][k] = updates[i][j][k] if axis = 0, output[i][indices[i][j][k]][k] = updates[i][j][k] if axis = 1, output[i][j][indices[i][j][k]] = updates[i][j][k] if axis = 2. Copy to clipboard Example for a 3-D tensor case is performed as below with *reduction* set to QNN\_OP\_SCATTER\_ELEMENTS\_REDUCTION\_ADD: output[indices[i][j][k]][j][k] += updates[i][j][k] if axis = 0, output[i][indices[i][j][k]][k] += updates[i][j][k] if axis = 1, output[i][j][indices[i][j][k]] += updates[i][j][k] if axis = 2. Copy to clipboard Example for a 3-D tensor case is performed as below with *reduction* set to QNN\_OP\_SCATTER\_ELEMENTS\_REDUCTION\_MUL: output[indices[i][j][k]][j][k] *= updates[i][j][k] if axis = 0, output[i][indices[i][j][k]][k] *= updates[i][j][k] if axis = 1, output[i][j][indices[i][j][k]] *= updates[i][j][k] if axis = 2. Copy to clipboard Example for a 3-D tensor case is performed as below with *reduction* set to QNN\_OP\_SCATTER\_ELEMENTS\_REDUCTION\_MAX: output[indices[i][j][k]][j][k] = max(output[indices[i][j][k]][j][k], updates[i][j][k]) if axis = 0, output[i][indices[i][j][k]][k] = max(output[i][indices[i][j][k]][k], updates[i][j][k]) if axis = 1, output[i][j][indices[i][j][k]] = max(output[i][j][indices[i][j][k]], updates[i][j][k]) if axis = 2. Copy to clipboard Note that for indices generated from other Operations (e.g. NonZero) we permit -1 to be provided as a value to indicate an index for ScatterElements Operation to skip/ignore. References: - ONNX: [ops::ScatterElements](https://github.com/onnx/onnx/blob/main/docs/Operators.md#ScatterElements) ### Inputs #### in[0] input data - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 #### in[1] indices : contains the index values used in the *axis* dimension to scatter *updates*. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32 - Shape: a tensor of rank N - Constraints: > > > - Value: Indices must be in range [0, Shape(in[0])[*axis*] - 1] #### in[2] updates : values to scatter into out[0]. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Same shape as in[1] > - Datatype: Same datatype as in[0] ### Parameters #### axis The axis of out[0] to scatter on. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Constraints: > > > - Value: Must be in range [0, N - 1] #### reduction Operation that is applied to all update values of in[2] to out[0] at the specified indices from in[1]. When *reduction* is set to “NONE”, values in in[2] are assigned to out[0] at the indices specified by in[1] and duplicate indices will override previous updated values. When *reduction* is set to “ADD”, values in in[2] are added to the values of out[0] at the indices specified by in[1] and duplicate indices are allowed. When *reduction* is set to “MUL”, values in in[2] are multiplied to the values of out[0] at the indices specified by in[1] and duplicate indices are allowed. When *reduction* is set to “MAX”, the maximum values between the values in in[2] and the values of out[0] are assigned to out[0] at the indices specified by in[1] and duplicate indices are allowed. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - NONE = 0, > - ADD = 1, > - MUL = 2, > - MAX = 3 ### Outputs #### out[0] Output data - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Same shape as in[0] > - Datatype: Same datatype as in[0] ## ScatterNd Takes three input tensors: data, indices, and updates to create an output tensor. out[0] is produced by creating a copy of in[0] and then updating it with values that are specified by in[2] at the index positions specified by in[1]. Note that for indices generated from other Operations (e.g. NonZero) we permit -1 to be provided as a value to indicate an index for ScatterNd Operation to skip/ignore. References: - ONNX: [ops::ScatterND](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ScatterND) - TensorFlow: [ops::scatter_nd](https://www.tensorflow.org/api_docs/python/tf/scatter_nd) ### Inputs #### in[0] input data - Mandatory: true - Data type: backend specific - Shape: a tensor of rank n - Constraints: > > > - Shape: Rank > 0 #### in[1] Tensor of indices to scatter updates to create output tensor. Note that when Shape(in[1])[-1] is equal to *n* each value of in[2] is specific to a single element of out[0]. Otherwise when Shape(in[1])[-1] is less than *n* each value of in[2] is specific to a slice of the output tensor. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32, QNN\_DATATYPE\_UINT\_32 - Shape: a tensor of rank k - Constraints: > > > - Shape: Rank > 0 > - Shape: Shape(in[1])[-1] must be >= 0 and <= n > - Value: Indices must be non-negative and within the range of the corresponding dimension of in[0] #### in[2] Updates to scatter into the output tensor - Mandatory: true - Data type: backend specific - Shape: a tensor of rank n + k - Shape(in[1])[-1] - 1, where *n* is the rank of in[0] and *k* is the rank of in[1]. - Constraints: > > > - Shape: Rank = n + k - Shape(in[1])[-1] - 1 > - Shape: The number of slices/elements to update should be equal to the number of indices provided. ### Parameters #### reduction Operation that is applied to all update values of in[2] to out[0] at the specified indices from in[1]. When *reduction* is set to “NONE”, values in in[2] are assigned to out[0] at the indices specified by in[1] and duplicate indices will override previous updated values. When *reduction* is set to “ADD”, values in in[2] are added to the values of out[0] at the indices specified by in[1] and duplicate indices are allowed. When *reduction* is set to “MUL”, values in in[2] are multiplied to the values of out[0] at the indices specified by in[1] and duplicate indices are allowed. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - NONE = 0, > - ADD = 1, > - MUL = 2 ### Outputs #### out[0] Output data - Mandatory: true - Data type: backend specific - Shape: a tensor of rank n - Constraints: > > > - Shape: Same shape as in[0] > - Datatype: Same datatype as in[2] ## Shape Generates a 1D output tensor containing the shape of the input tensor. Parameters *start* and *end* can be used to compute a slice of the input tensor’s shape. References: - ONNX: [ops::Shape](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Shape) - TensorFlow: [Shape](https://www.tensorflow.org/api_docs/python/tf/shape) ### Inputs #### in[0] - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension - Constraints: > > > - Shape: Rank > 0 ### Parameters #### start Starting axis for slicing the shape of the input tensor. Note that the start axis is inclusive and will include the size of the start axis in the output tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Constraints: > > > - Value: *start* must be in range [0, N-1] #### end Ending axis for slicing the shape of the input tensor. Note that the end axis specified is exclusive and will not include the size of the end axis in the output tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: N - Constraints: > > > - Value: *end* must be in range [*start* + 1, N] ### Outputs #### out[0] Shape of the input tensor. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [M] where M = *end* - *start* ## Sigmoid Computes the sigmoid activation function elementwise on an input tensor. The sigmoid function is defined as sigmoid(x) = 1 / (1 + exp(-x)) Copy to clipboard where *exp* is exponentiation by the base of the natural logarithm. Refer to Sigmoid backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NeuralNetworks: [ANEURALNETWORKS_LOGISTIC](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a82a340eb540933f638db420369650483) ### Inputs #### in[0] Input feature map. - Mandatory: true - Data type: backend specific - Shape: Any - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] Output feature map. - Mandatory: true - Data type: backend specific - Shape: Any - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(out[0])[i] must be dynamic. ## Softmax Computes data normalization exponentially on an input tensor given an optional positive scaling factor, beta. The computation is done element-wise per batch along the specified axis. Refer to Softmax backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_SOFTMAX](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a2bfbb83a537701e2843a3d5004250c2c) - TensorFlow: [Softmax](https://www.tensorflow.org/api_docs/python/tf/nn/softmax) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axis - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: N-1 - Constraints: > > > - Value: must be in range [0, N-1] #### beta - Mandatory: false - Data type: QNN\_DATATYPE\_FLOAT\_32 - Shape: scalar - Default: 1.0 ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: a tensor of N dimension - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Same shape as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then shape(out[0])[i] must be dynamic. ## SpaceToBatch A type of tensor realignment operation that rearranges blocks of spatial data in batch dimension. The op moves blocks of data of size (block\_size[0] \* block\_size[1]) from the height and width dimensions of the input tensor into the batch dimension of the output tensor after optional padding. References: - TensorFlow: [Space To Batch](https://www.tensorflow.org/api_docs/python/tf/space_to_batch) ### Inputs #### in[0] Input Activation. - Mandatory: true - Data type: backend specific - Shape: 4D tensor of shape [batch, height, width, depth] - Constraints: > > > - Shape: height must be divisible by block\_size[0] after any paddings have been applied. > - Shape: width must be divisible by block\_size[1] after any paddings have been applied. ### Parameters #### block\_size Vector that represents block size along the *height* and *width* dimensions respectively. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [block\_height, block\_width] - Constraints: > > > - Value: Elements must be >=1 #### pad\_amount Paddings along the height and width dimensions of the input tensor. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[pad\_top, pad\_bottom], [pad\_left, pad\_right]] - Default: [[0, 0], [0, 0]] ### Outputs #### out[0] Output Activation. Permuted output tensor with new spatial dimensions [*output\_height*, *output\_width*] defined by output_height = (height + pad_top + pad_bottom) / block_size[0] output_width = (width + pad_left + pad_right) / block_size[1] Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: 4D tensor of shape [(batch \* block\_size[0] \* block\_size[1]), output\_height, output\_width, depth] - Constraints: > > > - Datatype: Same datatype as in[0] ## SpaceToDepth A type of tensor realignment operation that rearranges blocks of spatial data into depth. The op moves blocks of data of size (block\_size[0] \* block\_size[1]) from the height and width dimensions of the input tensor into the depth dimension of the output tensor. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_SPACE_TO_DEPTH](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a90099ec472f6571a932b111d979dcccd) ### Inputs #### in[0] Input Activation. - Mandatory: true - Data type: backend specific - Shape: 4D tensor of shape [batch, height, width, depth] - Constraints: > > > - Shape: height must be divisible by block\_size[0] > - Shape: width must be divisible by block\_size[1] ### Parameters #### block\_size Vector that represents block size along the *height* and *width* dimensions respectively. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [block\_height, block\_width] - Constraints: > > > - Value: Elements must be >=1 #### mode Specifies the order in which elements of in[0] are rearranged. If QNN\_OP\_SPACE\_TO\_DEPTH\_PARAM\_MODE is set to QNN\_OP\_SPACE\_TO\_DEPTH\_MODE\_DCR then elements along the depth dimension are rearranged in the order of depth, column, and then row; if set to QNN\_OP\_SPACE\_TO\_DEPTH\_MODE\_CRD elements along the depth dimension are rearranged in the order of column, row, and then depth. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Values: > > > - DCR = 0, > - CRD = 1 ### Outputs #### out[0] Output Activation. - Mandatory: true - Data type: backend specific - Shape: 4D tensor of shape [batch, (height / block\_size[0]), (width / block\_size[1]), (depth \* block\_size[0] \* block\_size[1])] - Constraints: > > > - Datatype: Same datatype as in[0] ## SparseToDense Convert a sparse tensor to a dense tensor. References: - QNN: [CreateSparse](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#createsparse) - QNN: [GetSparseIndices](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#getsparseindices) - QNN: [GetSparseValues](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#getsparsevalues) ### Inputs #### in[0] input - Mandatory: true - Data type: backend specific - Shape: tensor of rank N - Constraints: > > > - Must be a sparse tensor. ### Parameters None ### Outputs #### out[0] output - Mandatory: true - Data type: backend specific - Shape: tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] > - Must not be a sparse tensor. > - Shape: Same shape as in[0] ## Split Splits input tensor along a given *axis* into multiple output tensors according to *split\_index*. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_SPLIT](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0afb787314a8631fe847f1fd93cfd576a7) - ONNX: [ops::Split](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Split) ### Inputs #### in[0] Input data - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axis Specifies axis to split on. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: must be in range [0, N-1] #### split\_index 1-D tensor specifying starting index of each split slice in *axis*. Index values must be in range [1, shape(in[0])[axis]-1] and split\_index[i+1] > split\_index[i]. *axis* is split into size(split\_index)+1 slices. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [M] - Constraints: > > > - Shape: M <= shape(in[0])[axis] > - Value: Must be in range [1, shape(in[0])[axis]-1] ### Outputs #### out[0..m] Resulting (M+1) output data tensors. This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: true - Data type: backend specific - Shape: rank(out[m]) = rank(in[0]), where sum(m, size(shape(out[m])[axis])) = size(shape(in[0])[axis]) - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same rank as in[0] ## Squeeze Removes dimensions of size 1 from the shape of the input tensor in[0]. The number of elements implied by output tensor must be the same as the input tensor. This functionality can also be achieved using the [Reshape](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#reshape) operation. Note that user can prevent removing certain static dimensions of size 1 by expressing as such in shape(out[0]) when axes is not provided. Refer to Squeeze backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_SQUEEZE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a1207019989837ee9d10c5b6663504933) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: Rank = N, N > 1 - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 1 ### Parameters #### axes Indices specifying the dimensions to squeeze. If axes is not provided all static dimensions of size 1 not specified in shape(out[0]) are removed. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: 1D of shape [K] - Default: None - Constraints: > > > - Value: Must be in range [0, N) > - Value: Must be unique > - Shape: shape(in[0])[axes[i]] must equal 1 ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: a tensor of rank M - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: M <= rank(in[0]) > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic then the corresponding dimension of out[0] after being squeezed must be dynamic. ## Stft Computes the Short-time Fourier Transform of the signal. References: - ONNX: [STFT](https://github.com/onnx/onnx/blob/main/docs/Operators.md#STFT) ### Inputs #### in[0] input signal - Mandatory: true - Data type: backend specific - Shape: [batch, signal\_length, 1] #### in[1] window : The window being slid over the signal. Note that the window is the size of each frame the STFT will process. - Mandatory: false - Data type: backend specific - Shape: [window\_size], where window\_size = frame\_length when not provided. - Default: {1,..,1} - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: window\_size must be equal to frame\_length. ### Parameters #### frame\_step The number of samples to step between successive DFTs (Discrete Fourier Transforms). - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be > 0 #### frame\_length The size of the DFT (Discrete Fourier Transform). - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: window\_size if window input is provided. Otherwise, signal\_length. - Constraints: > > > - Value: Must be <= signal\_length #### onesided When onesided is set to 1 : Only values for frequencies in the range [0, 1, 2, .., floor(n\_fft/2) + 1] are returned. Note this is based on the conjugate symmetry property of the fourier transform for real-valued signals, where X[m,w] = X[m,n\_fft - w]. When onesided is set to 0 : the complete representation of the frequency content of the signal both positive and negative frequencies are provided. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 ### Outputs #### out[0] output activation - Mandatory: true - Data type: backend specific - Shape: [batch, frames, frame\_length//2 + 1, 2] if onesided is set to 1. Otherwise, [batch, frames, frame\_length, 2]. Note frames = ((signal\_length - frame\_length)/frame\_step) + 1 - Constraints: > > > - Datatype: Same datatype as in[0] ## StridedSlice Extract “slices” from a tensor by sampling in each dimension. Parameters are applied in the following order: *new\_axes\_mask*, *ranges*/*end\_mask*/*begin\_mask*, and then *shrink\_axis*. *new\_axis\_mask* is applied to the input tensor to create an *intermediate tensor* and all other parameters are applied in respect to this *intermediate tensor*. If *new\_axis\_mask* is not provided then these parameters are applied directly to the input tensor. Equivalent to python: out = in[begin[0]:end[0]:stride[0],...,begin[n-1]:end[n-1]:stride[n-1]] Copy to clipboard Refer to StridedSlice backend definition for support of dynamic dimensions for each backend. Backends do not support dynamic dimensions unless stated otherwise. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_STRIDED_SLICE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a89695302f8b1e7ae7ce8f4d8c0b8a752) - TensorFlow: [strided_slice](https://www.tensorflow.org/api_docs/python/tf/strided_slice) ### Inputs #### in[0] Input data - Mandatory: true - Data type: backend specific - Shape: a tensor of rank n - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### ranges Specifies the slice range for each axis, in the form (begin,end,stride). All negative values for begin[i] and end[i] have shape(in[0])[i] added to them and then clipped to the following ranges depending on the stride value: When stride > 0: Begin values are clipped to the range [0, shape(in[0])[i]] and End values are clipped to the range [0, shape(in[0])[i]]. When stride < 0: Begin values are clipped to the range [0, shape(in[0])[i] - 1] and End values are clipped to the range [-1, shape(in[0])[i] - 1]. For a negative stride to include index 0, we permit the end of the range to be -1. Note that ranges extend “one past the end” similar to a C++ iterator, but the slice will include the begin index. Note when slicing a dynamic dimension to the end it is recommended to use INT\_MAX when slicing forward and INT\_MIN when slicing backwards. Negative strides are not supported for dynamic dimensions. - Mandatory: true - Data type: QNN\_DATATYPE\_INT\_32 - Shape: [rank(in[0]) + new,3] : [[begin\_0, end\_0, stride\_0], …. , [begin\_{n+new-1}, end\_{n+new-1}, stride\_{n+new-1}]], where *new* is the number of new axes that will be inserted. - Constraints: > > > - Value: Stride must be nonzero > - Value: Stride can only be negative if begin > end > - Value: Negative strides are not supported for dynamic dimensions #### begin\_mask A bit mask corresponding to the begin axes of ranges that indicates whether certain axes of the intermediate tensor are to be retained or ignored during the slicing. If the ith bit of begin\_mask is set, ranges[i][0] will be ignored and the fullest possible range in that dimension is used instead. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 #### end\_mask A bit mask corresponding to the end axes of ranges that indicates whether certain axes of the intermediate tensor are to be retained or ignored during the slicing. If the ith bit of end\_mask is set, ranges[i][1] will be ignored and the fullest possible range in that dimension is used instead. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 #### shrink\_axes A bit mask corresponding to the axes of the intermediate tensor to omit in the output shape. If the ith bit of *new\_axes\_mask* is also set it overrides any shrink\_axes setting at the same bit. All bits set beyond the shape of the intermediate tensor will be ignored. Note that the begin range provided for any axes that is omitted will be used in respect to the other axes. E.g. Given a 3-D shape [10, 20, 30] and only the 0th axis being omitted by shrink\_axes and the entire range is used for the other 2 axes. If range[0][0] = 5, which is the begin value for the 0th axis, then the output shape would be a 2-D shape of [20, 30] corresponding to the 6th slice of the 0th dimension. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Constraints: > > > - Value: When the ith bit of shrink\_axes\_mask is set and the ith bit of new\_axes\_mask is not set then the begin value at ranges[i][0] must be in the range of [-shape(in[0])[i], shape(in[0])[i] - 1] #### new\_axes\_mask A bit mask to insert new axes at positions specified by the set bits. If the ith bit in new\_axes\_mask is set, a new length 1 dimension is inserted into the input tensor at this position and an intermediate tensor is created. *ranges*, *end\_mask*, *begin\_mask* applied to positions where a new axes has been inserted are ignored since any new axes will always be size 1. Notes: - The most significant bit of new\_axes\_mask is defined by the number of *ranges* provided by the user, e.g. Given 8 sets of ranges the most significant bit would be the 7th bit. - From the 0th bit to the most significant bit in new\_axis\_mask there must be exactly rank(in[0]) number of unset bits. - All bits past the most significant bit will be ignored. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 ### Outputs #### out[0] Output data Dimension at axis i is max(1, ceil( (ranges[i,1]-ranges[i,0])/ranges[i,2] ) ), except where omitted by *shrink\_axes*, or ‘1’ where specified by *new\_axes\_mask*. - Mandatory: true - Data type: backend specific - Shape: rank is rank(in[0]) + new - shrink, where *new* is the number of new axes that will be inserted and *shrink* is the number of axes that will be omitted. - Dynamic Shape: All dimensions can be dynamic except for newly inserted dimensions determined by new\_axes\_mask. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic and either the corresponding bit of end\_mask is set or there is no slicing being done at dimension i then the corresponding dimension of out[0] must be dynamic. > - Dynamic Shape: For each dimension, if shape(in[0])[i] is dynamic and stride value at ranges[i][2] > 0 and begin value at ranges[i][0] >= 0 and either the end value provided at ranges[i][1] >= shape(in[0])[i] or is negative then the corresponding dimension of out[0] must be dynamic. ## Tanh Computes the hyperbolic tangent function elementwise over an input feature map. References: - Android NeuralNetworks: [ANEURALNETWORKS_TANH](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a4b63c9caab823f112d82d853a77381e5) ### Inputs #### in[0] Input feature map. - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Shape: Rank > 0 ### Parameters None ### Outputs #### out[0] Output feature map. - Mandatory: true - Data type: backend specific - Shape: Any - Constraints: > > > - Datatype: Same datatype as in[0] > - Shape: Same shape as in[0] ## Tile Creates a new tensor by tiling an input tensor. The input tensor is replicated along each dimension for as many times as specified by the multiples tensor. shape(out[0])[i] = shape(in[0])[i] * multiples[i] Copy to clipboard References: - Android NeuralNetworks: [ANEURALNETWORKS_TILE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0aae56dc75c058291cee272a537978ecd3) ### Inputs #### in[0] Input tensor - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters #### multiples A 1D tensor of length N which contains the replication factor for each dimension. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [N], N = rank(in[0]) ### Outputs #### out[0] The tiled output tensor shape(out[0])[i] = shape(in[0])[i] * multiples[i] Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Datatype: Same datatype as in[0] ## TopK Find the K largest or smallest values along the last dimension at each position in a tensor, and return those values sorted, along with their indices in the input tensor. References: - Android NeuralNetworks: [ANEURALNETWORKS_TOPK_V2](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0afb989792e22adaaa6fb540ae8415f562) ### Inputs #### in[0] Input tensor - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters #### k The number of elements to extract from the input tensor at each position. Must be <= shape(in[0])[rank(in[0])-1]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar #### largest Whether to return the top-K largest or smallest elements. If true, the largest elements are returned. Otherwise, the smallest elements are returned. - Mandatory: false - Data type: QNN\_DATATYPE\_BOOL\_8 - Shape: scalar - Default: 1 ### Outputs #### out[0] Sorted largest or smallest values of input tensor at each position. - Mandatory: true - Data type: backend specific - Shape: same as in[0], except the last dimension which is *k* - Constraints: > > > - Datatype: Same datatype as in[0] #### out[1] Index values of elements of out[0] in in[0]. If two elements of out[0] in the same position have the same value, the one with the larger index will appear first. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32, QNN\_DATATYPE\_INT\_32 - Shape: same as in[0], except the last dimension which is *k* - Constraints: > > > - Shape: Same shape as out[0] ## Transpose Transposes the input tensor to produce an output with the same data but axes permuted according to the perm tensor. Refer to Transpose backend definition per backend for support of dynamic dimensions. Backends do not support dynamic dimensions unless stated otherwise. shape(out[0])[i] = shape(in[0])[perm[i]] Copy to clipboard References: - Android NeuralNetworks: [ANEURALNETWORKS_TRANSPOSE](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a92d7bc95eb68525334b6cfe80cd271ee) ### Inputs #### in[0] Input tensor - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Shape: Rank > 0 ### Parameters #### perm The permutations of the dimensions of the input tensor. The tensor values should be in range [0,N-1] and each dimension must be listed only once. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [N], N = rank(in[0]) ### Outputs #### out[0] The permuted output tensor - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Dynamic Shape: All dimensions can be dynamic. - Constraints: > > > - Datatype: Same datatype as in[0] > - Dynamic Shape: For each dimension, if shape(in[0])[perm[i]] is dynamic, then shape(out[0])[i] must be dynamic. ## TransposeConv1d Performs the transpose 1D convolution operation. This operation is also known as “Deconvolution”. Application of the filter moves according to the specified stride. For backends supporting quantized data types, clients can pass filters which are either quantized per-tensor or per-axis with possible constraints on the axis value that is supported. For regular transpose convolution, *group* is 1. Group field greater than 1 implies a grouped transpose convolution where a group of different filters is applied to each output channel group and the result is concatenated together. References: - ONNX: [ConvTranspose](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ConvTranspose) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, width, channel\_in] - Constraints: > > > - Shape: channel\_in must be evenly divisible by group #### in[1] filters - Mandatory: true - Data type: backend specific - Shape: [filter\_width, channel\_in, channel\_out/group] - Constraints: > > > - Shape: channel\_out must be evenly divisible by group #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [channel\_out] - Default: {0,..,0} ### Parameters #### stride Defines stride for 1D spatial (i.e. width) axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: Must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 1D spatial (i.e. width) axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [width\_pad\_before, width\_pad\_after] #### group - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 #### output\_padding Controls the additional size added to the 1D spatial axes (i.e width) of the output shape. Note that output\_padding is only used to find output shape, but does not actually add zero-padding to output. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 0 - Constraints: > > > - Value: Must be < stride ### Outputs #### out[0] The output 1D spatial dimensions is a function of the filters, stride, and pad\_amount. width_out = floor(stride * (shape(in[0])[width] - 1) + shape(in[1])[width] - pad_amount[0] - pad_amount[1] + output_padding) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, width\_out, channel\_out] - Constraints: > > > - Datatype: Same datatype as in[0] ## TransposeConv2d Performs the transpose 2D convolution operation. This operation is also known as “Deconvolution”. Application of the filter moves according to the specified strides. For backends supporting quantized data types, clients can pass filters which are either quantized per-tensor or per-axis with possible constraints on the axis value that is supported. For regular transpose convolution, *group* is 1. Group field greater than 1 implies a grouped transpose convolution where a group of different filters is applied to each output channel group and the result is concatenated together. Note that *channel\_out* and *channel\_in* must be evenly divisible by *group*. Refer to TransposeConv2d backend definition for supported data type and layouts for each backend. References: - Android NDK NeuralNetworks: [ANEURALNETWORKS_TRANSPOSE_CONV_2D](https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a5ac65e772b035ddb27c18236752aa1fd) - ONNX: [ConvTranspose](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ConvTranspose) - QNN: [Conv2d](https://docs.qualcomm.com/doc/80-63442-50/topic/MasterOpDef.html#conv2d) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, height, width, channel\_in] #### in[1] filters - Mandatory: true - Data type: backend specific - Shape: [filter\_height, filter\_width, channel\_in, channel\_out/group] #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [channel\_out] - Default: [0] ### Parameters #### stride Defines stride for 2D spatial (i.e. height and width) axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] : [height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 2D spatial (i.e. height and width) axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2,2] with format [[height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### group - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 #### output\_padding Controls the additional size added to the 2D spatial axes (i.e height and width) of the output shape. Note that output\_padding is only used to find output shape, but does not actually add zero-padding to output. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [2] with format [height\_output\_padding, width\_output\_padding] - Default: [0, 0] - Constraints: > > > - Value: must be < corresponding strides dimension ### Outputs #### out[0] The output 2D spatial dimensions are functions of the filters, stride, and pad\_amount. height_out = floor(stride[0] * (shape(in[0])[height] - 1) + shape(in[1])[height] - pad_amount[0,0] - pad_amount[0,1] + output_padding[0]) width_out = floor(stride[1] * (shape(in[0])[width] - 1) + shape(in[1])[width] - pad_amount[1,0] - pad_amount[1,1] + output_padding[1]) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, height\_out, width\_out, channel\_out] - Constraints: > > > - Datatype: Same datatype as in[0] ## TransposeConv3d Performs the transpose 3D convolution operation. This operation is also known as “Deconvolution”. Application of the filter moves according to the specified strides. For backends supporting quantized data types, clients can pass filters which are either quantized per-tensor or per-axis with possible constraints on the axis value that is supported. For regular transpose convolution, *group* is 1. Group field greater than 1 implies a grouped transpose convolution where a group of different filters is applied to each output channel group and the result is concatenated together. Note that *channel\_out* and *channel\_in* must be evenly divisible by *group*. Refer to TransposeConv3d backend definition for supported data type and layouts for each backend. References: - TensorFlow: [conv3d_transpose](https://www.tensorflow.org/api_docs/python/tf/nn/conv3d_transpose) - ONNX: [ConvTranspose](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ConvTranspose) ### Inputs #### in[0] input activation - Mandatory: true - Data type: backend specific - Shape: [batch, depth, height, width, channel\_in] #### in[1] filters - Mandatory: true - Data type: backend specific - Shape: [filter\_depth, filter\_height, filter\_width, channel\_in, channel\_out/group] #### in[2] biases - Mandatory: false - Data type: backend specific - Shape: [channel\_out] - Default: [0] ### Parameters #### stride Defines stride for 3D spatial (i.e. depth, height and width) axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [depth\_stride, height\_stride, width\_stride] - Constraints: > > > - Value: Strides must be > 0 #### pad\_amount Pad amount to be added to the beginning and end part of 3D spatial (i.e. depth, height, and width) axes of in[0]. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3,2] with format [[depth\_pad\_before, depth\_pad\_after], [height\_pad\_before, height\_pad\_after], [width\_pad\_before, width\_pad\_after]] #### dilation Dilation value along each spatial axis (i.e. depth, height, and width) of the filter. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [depth\_dilation, height\_dilation, width\_dilation] - Default: [1, 1, 1] - Constraints: > > > - Value: Dilations must be > 0 #### group - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Default: 1 #### output\_padding Controls the additional size added to the 3D spatial axes (i.e depth, height, and width) of the output shape. Note that output\_padding is only used to find output shape, but does not actually add zero-padding to output. - Mandatory: false - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: [3] : [depth\_output\_padding, height\_output\_padding, width\_output\_padding] - Default: [0, 0, 0] - Constraints: > > > - Value: must be < corresponding strides dimension ### Outputs #### out[0] The output 3D spatial dimensions are functions of the *filters*, *stride*, and *pad\_amount*. dilated_filter_depth = (shape(in[1])[depth] - 1) * dilation[0] + 1 dilated_filter_height = (shape(in[1])[height] - 1) * dilation[1] + 1 dilated_filter_width = (shape(in[1])[width] - 1) * dilation[2] + 1 depth_out = floor(stride[0] * (shape(in[0])[depth] - 1) + dilated_filter_depth - pad_amount[0,0] - pad_amount[0,1] + output_padding[0]) height_out = floor(stride[1] * (shape(in[0])[height] - 1) + dilated_filter_height - pad_amount[1,0] - pad_amount[1,1] + output_padding[1]) width_out = floor(stride[2] * (shape(in[0])[width] - 1) + dilated_filter_width - pad_amount[2,0] - pad_amount[2,1] + output_padding[2]) Copy to clipboard - Mandatory: true - Data type: backend specific - Shape: [batch, depth\_out, height\_out, width\_out, channel\_out] - Constraints: > > > - Datatype: Same datatype as in[0] ## UnPack Unpacks input tensor along a given *axis* into shape(in[0])[axis] tensors with rank one lower than in[0] by chipping it along the axis dimension. References: - TensorFlow: [unstack](https://www.tensorflow.org/api_docs/python/tf/unstack) ### Inputs #### in[0] Input tensor. - Mandatory: true - Data type: backend specific - Shape: a tensor of rank N - Constraints: > > > - Shape: Rank > 0 ### Parameters #### axis Specifies axis to unpack on. - Mandatory: true - Data type: QNN\_DATATYPE\_UINT\_32 - Shape: scalar - Constraints: > > > - Value: must be in range [0, N-1] ### Outputs #### out[0..m] Resulting m = shape(in[0])[axis] output data tensors. This tensor is repeated, meaning the same definition can apply to multiple tensors. - Mandatory: true - Data type: backend specific - Shape: rank(out[0..m]) = rank(in[0]) - 1 - Constraints: > > > - Datatype: Same datatype as in[0] Last Published: Oct 10, 2025 [Previous Topic Operation configuration and tensor sizing notes](https://docs.qualcomm.com/bundle/publicresource/80-63442-50/topics/terminology.md) [Next Topic Block operators](https://docs.qualcomm.com/bundle/publicresource/80-63442-50/topics/operations.md)