# Tools This page describes the SDK tools and features for Linux/Android and Windows developers. | Category | Tool | Developer | Developer | Developer | Developer | Developer | Developer | | --- | --- | --- | --- | --- | --- | --- | --- | | Category | Tool | Linux/Android | Linux/Android | Linux/Android | Windows | Windows | Windows | | Category | Tool | Ubuntu | WSL x86\_64 (\*) | Device | WSL x86\_64 (\*) | Windows x86\_64 | Windows on Snapdragon | | Model Conversion | [snpe-onnx-to-dlc](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-onnx-to-dlc) | Yes | Yes | | Yes | Yes | Yes\*\* | | Model Conversion | [snpe-pytorch-to-dlc](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-pytorch-to-dlc) | Yes | Yes | | Yes | | | | Model Conversion | [snpe-tensorflow-to-dlc](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-tensorflow-to-dlc) | Yes | Yes | | Yes | Yes | Yes\*\* | | Model Conversion | [snpe-tflite-to-dlc](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-tflite-to-dlc) | Yes | Yes | | Yes | | | | Model Conversion | [qairt-converter](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#qairt-converter) | Yes | Yes | | Yes | Yes | Yes\*\* | | Model Preparation | [snpe-dlc-graph-prepare](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-dlc-graph-prepare) | Yes | Yes | | Yes | Yes | Yes\*\* | | Model Preparation | [snpe-dlc-quant](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-dlc-quant) | Yes | Yes | | Yes | Yes | Yes | | Model Preparation | [snpe-dlc-quantize](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-dlc-quantize) | Yes | Yes | | Yes | | | | Model Preparation | [snpe-udo-package-generator](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-udo-package-generator) | Yes | Yes | | Yes | | | | Model Preparation | [qairt-quantizer](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#qairt-quantizer) | Yes | Yes | | Yes | Yes | Yes | | Execution | [snpe-net-run](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-net-run) | Yes | Yes | Yes | Yes | Yes | Yes | | Execution | [snpe-parallel-run](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-parallel-run) | Yes | Yes | Yes | Yes | | Yes | | Execution | [snpe-throughput-net-run](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-throughput-net-run) | Yes | Yes | Yes | Yes | Yes | Yes | | Analysis | [snpe-diagview](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-diagview) | Yes | Yes | | | Yes | Yes | | Analysis | [snpe-dlc-diff](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-dlc-diff) | Yes | Yes | | Yes | Yes | Yes\*\* | | Analysis | [snpe-dlc-info](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-dlc-info) | Yes | Yes | | Yes | Yes | Yes\*\* | | Analysis | [snpe-dlc-viewer](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-dlc-viewer) | Yes | Yes | | Yes | Yes | Yes\*\* | | Analysis | [snpe-platform-validator](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-platform-validator) | | | Yes | | | Yes | | Analysis | [snpe-platform-validator-py](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-platform-validator-py) | Yes | | | | | | | Analysis | [snpe_bench.py](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-bench-py) | Yes | Yes | | | | | | Analysis | [qairt-dlc-diff](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#qairt-dlc-diff) | Yes | Yes | | Yes | Yes | Yes\*\* | | Analysis | [qairt-dlc-info](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#qairt-dlc-info) | Yes | Yes | | Yes | Yes | Yes\*\* | | Analysis | [Architecture Checker (Experimental)](https://docs.qualcomm.com/doc/80-63442-2/topic/debug_tools_snpe-architecture-checker.html) | Yes | Yes | | Yes | Yes | Yes\*\* | | Analysis | [Accuracy Debugger (Experimental)](https://docs.qualcomm.com/doc/80-63442-2/topic/debug_tools_snpe-accuracy-debugger.html) | Yes | | | | Yes\*\*\* | Yes | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note \* Binary used in Windows WSL x86\_64 is from Ubuntu, related installation documentation please reference to [Qualcomm (R) Neural Processing SDK Setup](https://docs.qualcomm.com/doc/80-63442-2/topic/setup.html) \* For ARM64X in Windows, only snpe-net-run and snpe-throughput-net-run are supported. \*\* Requires the python scripts and the executables from the Windows x86\_64 binary folder. \*\*\* snpe-accuracy-debugger on Windows x86 system is tested only for CPU runtime currently. Note \* When using converter tools in Windows PowerShell, make sure a virtual environment with the required packages is actived and converters are executed via **python**, as shown in the following example. (venv-3.10) > python snpe-onnx-to-dlc <options> Note - TFlite conversion using [qairt-converter](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#qairt-converter) is not supported for Windows x86\_64 and Windows on Snapdragon due to TVM library dependency. ## Model Conversion ### snpe-onnx-to-dlc snpe-onnx-to-dlc converts a serialized ONNX model into a DLC file. Current ONNX Conversion supports upto ONNX Opset 22. usage: snpe-onnx-to-dlc [--out_node OUT_NAMES] [--input_type INPUT_NAME INPUT_TYPE] [--input_dtype INPUT_NAME INPUT_DTYPE] [--input_encoding [...]] [--input_layout INPUT_NAME INPUT_LAYOUT] [--custom_io CUSTOM_IO] [--preserve_io [PRESERVE_IO ...]] [--dump_qairt_io_config_yaml [DUMP_QAIRT_IO_CONFIG_YAML]] [--enable_framework_trace] [--dry_run [DRY_RUN]] [-d INPUT_NAME INPUT_DIM] [-n] [-b BATCH] [-s SYMBOL_NAME VALUE] [--dump_custom_io_config_template DUMP_CUSTOM_IO_CONFIG_TEMPLATE] [--quantization_overrides QUANTIZATION_OVERRIDES] [--keep_quant_nodes] [--disable_batchnorm_folding] [--expand_lstm_op_structure] [--keep_disconnected_nodes] [--preserve_onnx_output_order] [--apply_masked_softmax {compressed,uncompressed}] [--packed_masked_softmax_inputs PACKED_MASKED_SOFTMAX_INPUTS [PACKED_MASKED_SOFTMAX_INPUTS ...]] [--packed_max_seq PACKED_MAX_SEQ] --input_network INPUT_NETWORK [-h] [--debug [DEBUG]] [-o OUTPUT_PATH] [--copyright_file COPYRIGHT_FILE] [--float_bitwidth FLOAT_BITWIDTH] [--float_bw FLOAT_BW] [--float_bias_bitwidth FLOAT_BIAS_BITWIDTH] [--float_bias_bw FLOAT_BIAS_BW] [--model_version MODEL_VERSION] [--validation_target RUNTIME_TARGET PROCESSOR_TARGET] [--strict] [--udo_config_paths CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] [--op_package_lib OP_PACKAGE_LIB] [--converter_op_package_lib CONVERTER_OP_PACKAGE_LIB] [-p PACKAGE_NAME | --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] Script to convert ONNX model into DLC required arguments: --input_network INPUT_NETWORK, -i INPUT_NETWORK Path to the source framework model. optional arguments: --out_node OUT_NAMES, --out_name OUT_NAMES Name of the graph's output Tensor Names. Multiple output names should be provided separately like: --out_name out_1 --out_name out_2 --input_type INPUT_NAME INPUT_TYPE, -t INPUT_NAME INPUT_TYPE Type of data expected by each input op/layer. Type for each input is |default| if not specified. For example: "data" image.Note that the quotes should always be included in order to handle special characters, spaces,etc. For multiple inputs specify multiple --input_type on the command line. Eg: --input_type "data1" image --input_type "data2" opaque These options get used by DSP runtime and following descriptions state how input will be handled for each option. Image: Input is float between 0-255 and the input's mean is 0.0f and the input's max is 255.0f. We will cast the float to uint8ts and pass the uint8ts to the DSP. Default: Pass the input as floats to the dsp directly and the DSP will quantize it. Opaque: Assumes input is float because the consumer layer(i.e next layer) requires it as float, therefore it won't be quantized. Choices supported: image default opaque --input_dtype INPUT_NAME INPUT_DTYPE The names and datatype of the network input layers specified in the format [input_name datatype], for example: 'data' 'float32' Default is float32 if not specified Note that the quotes should always be included in order to handlespecial characters, spaces, etc. For multiple inputs specify multiple --input_dtype on the command line like: --input_dtype 'data1' 'float32' --input_dtype 'data2' 'float32' --input_encoding INPUT_ENCODING [INPUT_ENCODING ...], -e INPUT_ENCODING [INPUT_ENCODING ...] Usage: --input_encoding "INPUT_NAME" INPUT_ENCODING_IN [INPUT_ENCODING_OUT] Input encoding of the network inputs. Default is bgr. e.g. --input_encoding "data" rgba Quotes must wrap the input node name to handle special characters, spaces, etc. To specify encodings for multiple inputs, invoke --input_encoding for each one. e.g. --input_encoding "data1" rgba --input_encoding "data2" other Optionally, an output encoding may be specified for an input node by providing a second encoding. The default output encoding is bgr. e.g. --input_encoding "data3" rgba rgb Input encoding types: image color encodings: bgr,rgb, nv21, nv12, ... time_series: for inputs of rnn models; other: not available above or is unknown. Supported encodings: bgr rgb rgba argb32 nv21 nv12 time_series other --input_layout INPUT_NAME INPUT_LAYOUT, -l INPUT_NAME INPUT_LAYOUT Layout of each input tensor. If not specified, it will use the default based on the Source Framework, shape of input and input encoding. Accepted values are- NCDHW, NDHWC, NCHW, NHWC, HWIO, OIHW, NFC, NCF, NTF, TNF, NF, NC, F, NONTRIVIAL N = Batch, C = Channels, D = Depth, H = Height, W = Width, F = Feature, T = Time NDHWC/NCDHW used for 5d inputs NHWC/NCHW used for 4d image-like inputs NFC/NCF used for inputs to Conv1D or other 1D ops NTF/TNF used for inputs with time steps like the ones used for LSTM op NF used for 2D inputs, like the inputs to Dense/FullyConnected layers NC used for 2D inputs with 1 for batch and other for Channels (rarely used) F used for 1D inputs, e.g. Bias tensor NONTRIVIAL for everything elseFor multiple inputs specify multiple --input_layout on the command line. Eg: --input_layout "data1" NCHW --input_layout "data2" NCHW --custom_io CUSTOM_IO Use this option to specify a yaml file for custom IO. --preserve_io [PRESERVE_IO [PRESERVE_IO ...]] Use this option to preserve IO layout and datatype. The different ways of using this option are as follows: --preserve_io layout --preserve_io datatype In this case, user should also specify the string - layout or datatype in the command to indicate that converter needs to preserve the layout or datatype. e.g. --preserve_io layout input1 input2 output1 --preserve_io datatype input1 input2 output1 Optionally, the user may choose to preserve the layout and/or datatype for all the inputs and outputs of the graph. This can be done in the following two ways: --preserve_io layout --preserve_io datatype Additionally, the user may choose to preserve both layout and datatypes for all IO tensors by just passing the option as follows: --preserve_io Note: Only one of the above usages are allowed at a time. Note: --custom_io gets higher precedence than --preserve_io. --dump_qairt_io_config_yaml [DUMP_QAIRT_IO_CONFIG_YAML] Use this option to dump a yaml file which contains the equivalent I/O configurations of QAIRT Converter along with the QAIRT Converter Command and can be passed to QAIRT Converter using the option --io_config. --enable_framework_trace Use this option to enable converter to trace the op/tensor change information. Currently framework op trace is supported only for ONNX converter. --dry_run [DRY_RUN] Evaluates the model without actually converting any ops, and returns unsupported ops/attributes as well as unused inputs and/or outputs if any. Leave empty or specify "info" to see dry run as a table, or specify "debug" to show more detailed messages only" -d INPUT_NAME INPUT_DIM, --input_dim INPUT_NAME INPUT_DIM The name and dimension of all the input buffers to the network specified in the format [input_name comma-separated-dimensions], for example: 'data' 1,224,224,3. Note that the quotes should always be included in order to handle special characters, spaces, etc. For scalar inputs, use a single dimension `0` to indicate that the input is a scalar value. For multiple inputs specify multiple --input_dim on the command line like: --input_dim 'data1' 1,224,224,3 --input_dim 'data2' 0 NOTE: This feature works only with Onnx 1.6.0 and above -n, --no_simplification Do not attempt to simplify the model automatically. This may prevent some models from properly converting when sequences of unsupported static operations are present. -b BATCH, --batch BATCH The batch dimension override. This will take the first dimension of all inputs and treat it as a batch dim, overriding it with the value provided here. For example: --batch 6 will result in a shape change from [1,3,224,224] to [6,3,224,224]. If there are inputs without batch dim this should not be used and each input should be overridden independently using -d option for input dimension overrides. -s SYMBOL_NAME VALUE, --define_symbol SYMBOL_NAME VALUE This option allows overriding specific input dimension symbols. For instance you might see input shapes specified with variables such as : data: [1,3,height,width] To override these simply pass the option as: --define_symbol height 224 --define_symbol width 448 which results in dimensions that look like: data: [1,3,224,448] --dump_custom_io_config_template DUMP_CUSTOM_IO_CONFIG_TEMPLATE Dumps the yaml template for Custom I/O configuration. This file canbe edited as per the custom requirements and passed using the option --custom_ioUse this option to specify a yaml file to which the custom IO config template is dumped. --disable_batchnorm_folding --expand_lstm_op_structure Enables optimization that breaks the LSTM op to equivalent math ops --keep_disconnected_nodes Disable Optimization that removes Ops not connected to the main graph. This optimization uses output names provided over commandline OR inputs/outputs extracted from the Source model to determine the main graph --preserve_onnx_output_order Preserve the ONNX output order in the converted graph. Note: This may slightly impact performance. -h, --help show this help message and exit --debug [DEBUG] Run the converter in debug mode. -o OUTPUT_PATH, --output_path OUTPUT_PATH Path where the converted Output model should be saved.If not specified, the converter model will be written to a file with same name as the input model --copyright_file COPYRIGHT_FILE Path to copyright file. If provided, the content of the file will be added to the output model. --float_bitwidth FLOAT_BITWIDTH Use the --float_bitwidth option to convert the graph to the specified float bitwidth, either 32 (default) or 16. --float_bw FLOAT_BW Note: --float_bw is deprecated, use --float_bitwidth. --float_bias_bitwidth FLOAT_BIAS_BITWIDTH Use the --float_bias_bitwidth option to select the bitwidth to use for float bias tensor --float_bias_bw FLOAT_BIAS_BW Note: --float_bias_bw is deprecated, use --float_bias_bitwidth. --model_version MODEL_VERSION User-defined ASCII string to identify the model, only first 64 bytes will be stored --validation_target RUNTIME_TARGET PROCESSOR_TARGET Note: This option is deprecated. A combination of processor and runtime target against which model will be validated. Choices for RUNTIME_TARGET: {cpu, gpu, dsp}. Choices for PROCESSOR_TARGET: {snapdragon_801, snapdragon_820, snapdragon_835}. If not specified, will validate model against {snapdragon_820, snapdragon_835} across all runtime targets. --strict Note: This option is deprecated. If specified, will validate in strict mode whereby model will not be produced if it violates constraints of the specified validation target. If not specified, will validate model in permissive mode against the specified validation target. --udo_config_paths CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...], -udo CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...] Path to the UDO configs (space separated, if multiple) Custom Op Package Options: --op_package_lib OP_PACKAGE_LIB, -opl OP_PACKAGE_LIB Use this argument to pass an op package library for quantization. Must be in the form and be separated by a comma for multiple package libs --converter_op_package_lib CONVERTER_OP_PACKAGE_LIB, -cpl CONVERTER_OP_PACKAGE_LIB Absolute path to converter op package library compiled by the OpPackage generator. Must be separated by a comma for multiple package libraries. Note: Order of converter op package libraries must follow the order of xmls. Ex1: --converter_op_package_lib absolute_path_to/libExample.so Ex2: -cpl absolute_path_to/libExample1.so,absolute_path_to/libExample2.so -p PACKAGE_NAME, --package_name PACKAGE_NAME A global package name to be used for each node in the Model.cpp file. Defaults to Qnn header defined package name --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...], -opc CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...] Path to a Qnn Op Package XML configuration file that contains user defined custom operations. Quantizer Options: --quantization_overrides QUANTIZATION_OVERRIDES Use this option to specify a json file with parameters to use for quantization. These will override any quantization data carried from conversion (eg TF fake quantization) or calculated during the normal quantization process. Format defined as per AIMET specification. --keep_quant_nodes Use this option to keep activation quantization nodes in the graph rather than stripping them. Masked Softmax Optimization Options: --apply_masked_softmax {compressed,uncompressed} This flag enables the pass that creates a MaskedSoftmax Op and rewrites the graph to include this Op. MaskedSoftmax Op may not be supported by all the QNN backends. Please check the supplemental backend XML for the targeted backend. This argument takes a string parameter input that selects the mode of MaskedSoftmax Op. 'compressed' value rewrites the graph with the compressed version of MaskedSoftmax Op. 'uncompressed' value rewrites the graph with the uncompressed version of MaskedSoftmax Op. --packed_masked_softmax_inputs PACKED_MASKED_SOFTMAX_INPUTS [PACKED_MASKED_SOFTMAX_INPUTS ...] Mention the input ids tensor name which will be packed in the single inference. This is applicable only for Compressed MaskedSoftmax Op. This will create a new input to the graph named 'position_ids' with same shape as the provided input name in this flag. During runtime, this input shall be provided with the token locations for individual sequences so that the same will be internally passed to positional embedding layer. E.g. If 2 sequences of length 20 and 30 are packed together in single batch of 64 tokens then this new input 'position_ids' should have value [0, 1, ..., 19, 0, 1, ..., 29, 0, 0, 0, ..., 0] Usage: --packed_masked_softmax input_ids Packed model will enable the user to pack multiple sequences into single batch of inference. --packed_max_seq PACKED_MAX_SEQ Number of sequences packed in the single input ids and single attention mask inputs. Applicable only for Compressed MaskedSoftmax Op. Note: Only one of: {'op_package_config', 'package_name'} can be specified Note: Only one of: {'op_package_config', 'package_name'} can be specified Copy to clipboard For more information, see [ONNX Model Conversion](https://docs.qualcomm.com/doc/80-63442-2/topic/model_conv_onnx.html) Additional details: > > > - *input\_layout argument:* > > > > > > > > > - Used with TF2Onnx or Keras2Onnx models when the input layout is NHWC. > > Onnx converter assumes that 4D inputs to the model are used by CNNs and are in NCHW format. > > For Keras2Onnx or TF2Onnx models, where the input is NHWC followed most likely by a transpose > > to NCHW, the converter will fail to successfully convert and optimize the model without the > > use of this argument. ### snpe-pytorch-to-dlc snpe-pytorch-to-dlc converts a serialized PyTorch model into a DLC file. usage: snpe-pytorch-to-dlc -d INPUT_NAME INPUT_DIM [--out_node OUT_NAMES] [--input_type INPUT_NAME INPUT_TYPE] [--input_dtype INPUT_NAME INPUT_DTYPE] [--input_encoding ...] [--input_layout INPUT_NAME INPUT_LAYOUT] [--custom_io CUSTOM_IO] [--preserve_io [PRESERVE_IO [PRESERVE_IO ...]]] [--dump_relay DUMP_RELAY] [--quantization_overrides QUANTIZATION_OVERRIDES] [--keep_quant_nodes] [--disable_batchnorm_folding] [--expand_lstm_op_structure] [--keep_disconnected_nodes] --input_network INPUT_NETWORK [-h] [--debug [DEBUG]] [-o OUTPUT_PATH] [--copyright_file COPYRIGHT_FILE] [--float_bitwidth FLOAT_BITWIDTH] [--float_bw FLOAT_BW] [--float_bias_bw FLOAT_BIAS_BW] [--model_version MODEL_VERSION] [--validation_target RUNTIME_TARGET PROCESSOR_TARGET] [--strict] [--udo_config_paths CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] [--op_package_lib OP_PACKAGE_LIB] [--converter_op_package_lib CONVERTER_OP_PACKAGE_LIB] [-p PACKAGE_NAME | --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] Script to convert PyTorch model into DLC required arguments: -d INPUT_NAME INPUT_DIM, --input_dim INPUT_NAME INPUT_DIM The names and dimensions of the network input layers specified in the format [input_name comma-separated-dimensions], for example: 'data' 1,3,224,224 Note that the quotes should always be included in order to handle special characters, spaces, etc. For multiple inputs specify multiple --input_dim on the command line like: --input_dim 'data1' 1,3,224,224 --input_dim 'data2' 1,50,100,3 --input_network INPUT_NETWORK, -i INPUT_NETWORK Path to the source framework model. optional arguments: --out_node OUT_NAMES, --out_name OUT_NAMES Name of the graph's output Tensor Names. Multiple output names should be provided separately like: --out_name out_1 --out_name out_2 --input_type INPUT_NAME INPUT_TYPE, -t INPUT_NAME INPUT_TYPE Type of data expected by each input op/layer. Type for each input is |default| if not specified. For example: "data" image.Note that the quotes should always be included in order to handle special characters, spaces,etc. For multiple inputs specify multiple --input_type on the command line. Eg: --input_type "data1" image --input_type "data2" opaque These options get used by DSP runtime and following descriptions state how input will be handled for each option. Image: Input is float between 0-255 and the input's mean is 0.0f and the input's max is 255.0f. We will cast the float to uint8ts and pass the uint8ts to the DSP. Default: Pass the input as floats to the dsp directly and the DSP will quantize it. Opaque: Assumes input is float because the consumer layer(i.e next layer) requires it as float, therefore it won't be quantized. Choices supported: image default opaque --input_dtype INPUT_NAME INPUT_DTYPE The names and datatype of the network input layers specified in the format [input_name datatype], for example: 'data' 'float32' Default is float32 if not specified. Note that the quotes should always be included in order to handle special characters, spaces, etc. For multiple inputs specify multiple --input_dtype on the command line like: --input_dtype 'data1' 'float32' --input_dtype 'data2' 'float32' --input_encoding INPUT_ENCODING [INPUT_ENCODING ...], -e INPUT_ENCODING [INPUT_ENCODING ...] Usage: --input_encoding "INPUT_NAME" INPUT_ENCODING_IN [INPUT_ENCODING_OUT] Input encoding of the network inputs. Default is bgr. e.g. --input_encoding "data" rgba Quotes must wrap the input node name to handle special characters, spaces, etc. To specify encodings for multiple inputs, invoke --input_encoding for each one. e.g. --input_encoding "data1" rgba --input_encoding "data2" other Optionally, an output encoding may be specified for an input node by providing a second encoding. The default output encoding is bgr. e.g. --input_encoding "data3" rgba rgb Input encoding types: image color encodings: bgr,rgb, nv21, nv12, ... time_series: for inputs of rnn models; other: not available above or is unknown. Supported encodings: bgr rgb rgba argb32 nv21 nv12 time_series other --input_layout INPUT_NAME INPUT_LAYOUT, -l INPUT_NAME INPUT_LAYOUT Layout of each input tensor. If not specified, it will use the default based on the Source Framework, shape of input and input encoding. Accepted values are- NCDHW, NDHWC, NCHW, NHWC, NFC, NCF, NTF, TNF, NF, NC, F, NONTRIVIAL N = Batch, C = Channels, D = Depth, H = Height, W = Width, F = Feature, T = Time NDHWC/NCDHW used for 5d inputs NHWC/NCHW used for 4d image-like inputs NFC/NCF used for inputs to Conv1D or other 1D ops NTF/TNF used for inputs with time steps like the ones used for LSTM op NF used for 2D inputs, like the inputs to Dense/FullyConnected layers NC used for 2D inputs with 1 for batch and other for Channels (rarely used) F used for 1D inputs, e.g. Bias tensor NONTRIVIAL for everything elseFor multiple inputs specify multiple --input_layout on the command line. Eg: --input_layout "data1" NCHW --input_layout "data2" NCHW Note: This flag does not set the layout of the input tensor in the converted DLC. --custom_io CUSTOM_IO Use this option to specify a yaml file for custom IO. --preserve_io [PRESERVE_IO [PRESERVE_IO ...]] Use this option to preserve IO layout and datatype. The different ways of using this option are as follows: --preserve_io layout --preserve_io datatype In this case, user should also specify the string - layout or datatype in the command to indicate that converter needs to preserve the layout or datatype. e.g. --preserve_io layout input1 input2 output1 --preserve_io datatype input1 input2 output1 Optionally, the user may choose to preserve the layout and/or datatype for all the inputs and outputs of the graph. This can be done in the following two ways: --preserve_io layout --preserve_io datatype Additionally, the user may choose to preserve both layout and datatypes for all IO tensors by just passing the option as follows: --preserve_io Note: Only one of the above usages are allowed at a time. Note: --custom_io gets higher precedence than --preserve_io. --dump_relay DUMP_RELAY Dump Relay ASM and Params at the path provided with the argument Usage: --dump_relay --disable_batchnorm_folding --expand_lstm_op_structure Enables optimization that breaks the LSTM op to equivalent math ops --keep_disconnected_nodes Disable Optimization that removes Ops not connected to the main graph. This optimization uses output names provided over commandline OR inputs/outputs extracted from the Source model to determine the main graph -h, --help show this help message and exit -o OUTPUT_PATH, --output_path OUTPUT_PATH Path where the converted Output model should be saved.If not specified, the converter model will be written to a file with same name as the input model --copyright_file COPYRIGHT_FILE Path to copyright file. If provided, the content of the file will be added to the output model. --float_bitwidth FLOAT_BITWIDTH Use the --float_bitwidth option to select the bitwidth to use when using float for parameters(weights/bias) and activations for all ops or specific Op (via encodings) selected through encoding, either 32 (default) or 16. --float_bw FLOAT_BW Note: --float_bw is deprecated, use --float_bitwidth. --float_bias_bw FLOAT_BIAS_BW Use the --float_bias_bw option to select the bitwidth to use for the float bias tensor --model_version MODEL_VERSION User-defined ASCII string to identify the model, only first 64 bytes will be stored --validation_target RUNTIME_TARGET PROCESSOR_TARGET A combination of processor and runtime target against which model will be validated. Choices for RUNTIME_TARGET: {cpu, gpu, dsp}. Choices for PROCESSOR_TARGET: {snapdragon_801, snapdragon_820, snapdragon_835}. If not specified, will validate model against {snapdragon_820, snapdragon_835} across all runtime targets. --strict If specified, will validate in strict mode whereby model will not be produced if it violates constraints of the specified validation target. If not specified, will validate model in permissive mode against the specified validation target. --udo_config_paths UDO_CONFIG_PATHS [UDO_CONFIG_PATHS ...], -udo UDO_CONFIG_PATHS [UDO_CONFIG_PATHS ...] Path to the UDO configs (space separated, if multiple) Quantizer Options: --quantization_overrides QUANTIZATION_OVERRIDES Use this option to specify a json file with parameters to use for quantization. These will override any quantization data carried from conversion (eg TF fake quantization) or calculated during the normal quantization process. Format defined as per AIMET specification. --keep_quant_nodes Use this option to keep activation quantization nodes in the graph rather than stripping them. Custom Op Package Options: --op_package_lib OP_PACKAGE_LIB, -opl OP_PACKAGE_LIB Use this argument to pass an op package library for quantization. Must be in the form and be separated by a comma for multiple package libs --converter_op_package_lib CONVERTER_OP_PACKAGE_LIB, -cpl CONVERTER_OP_PACKAGE_LIB Path to converter op package library compiled by the OpPackage generator. -p PACKAGE_NAME, --package_name PACKAGE_NAME A global package name to be used for each node in the Model.cpp file. Defaults to Qnn header defined package name --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...], -opc CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...] Path to a Qnn Op Package XML configuration file that contains user defined custom operations. Note: Only one of: {'package_name', 'op_package_config'} can be specified Copy to clipboard For more information, see [PyTorch Model Conversion](https://docs.qualcomm.com/doc/80-63442-2/topic/model_conv_pytorch.html) ### snpe-tensorflow-to-dlc snpe-tensorflow-to-dlc converts a TensorFlow model into a DLC file. usage: snpe-tensorflow-to-dlc -d INPUT_NAME INPUT_DIM --out_node OUT_NAMES [--input_type INPUT_NAME INPUT_TYPE] [--input_dtype INPUT_NAME INPUT_DTYPE] [--input_encoding ...] [--input_layout INPUT_NAME INPUT_LAYOUT] [--custom_io CUSTOM_IO] [--preserve_io [PRESERVE_IO [PRESERVE_IO ...]]] [--show_unconsumed_nodes] [--saved_model_tag SAVED_MODEL_TAG] [--saved_model_signature_key SAVED_MODEL_SIGNATURE_KEY] [--quantization_overrides QUANTIZATION_OVERRIDES] [--keep_quant_nodes] [--disable_batchnorm_folding] [--expand_lstm_op_structure] [--keep_disconnected_nodes] --input_network INPUT_NETWORK [-h] [--debug [DEBUG]] [-o OUTPUT_PATH] [--copyright_file COPYRIGHT_FILE] [--float_bitwidth FLOAT_BITWIDTH] [--float_bw FLOAT_BW] [--float_bias_bw FLOAT_BIAS_BW] [--model_version MODEL_VERSION] [--validation_target RUNTIME_TARGET PROCESSOR_TARGET] [--strict] [--udo_config_paths CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] [--op_package_lib OP_PACKAGE_LIB] [--converter_op_package_lib CONVERTER_OP_PACKAGE_LIB] [-p PACKAGE_NAME | --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] Script to convert TF model into DLC. required arguments: -d INPUT_NAME INPUT_DIM, --input_dim INPUT_NAME INPUT_DIM The names and dimensions of the network input layers specified in the format [input_name comma-separated-dimensions], for example: 'data' 1,224,224,3 Note that the quotes should always be included in order to handle special characters, spaces, etc. For multiple inputs specify multiple --input_dim on the command line like: --input_dim 'data1' 1,224,224,3 --input_dim 'data2' 1,50,100,3 --out_node OUT_NAMES, --out_name OUT_NAMES Name of the graph's output Tensor Names. Multiple output names should be provided separately like: --out_name out_1 --out_name out_2 --input_network INPUT_NETWORK, -i INPUT_NETWORK Path to the source framework model. optional arguments: --input_type INPUT_NAME INPUT_TYPE, -t INPUT_NAME INPUT_TYPE Type of data expected by each input op/layer. Type for each input is |default| if not specified. For example: "data" image.Note that the quotes should always be included in order to handle special characters, spaces,etc. For multiple inputs specify multiple --input_type on the command line. Eg: --input_type "data1" image --input_type "data2" opaque These options get used by DSP runtime and following descriptions state how input will be handled for each option. Image: Input is float between 0-255 and the input's mean is 0.0f and the input's max is 255.0f. We will cast the float to uint8ts and pass the uint8ts to the DSP. Default: Pass the input as floats to the dsp directly and the DSP will quantize it. Opaque: Assumes input is float because the consumer layer(i.e next layer) requires it as float, therefore it won't be quantized. Choices supported: image default opaque --input_dtype INPUT_NAME INPUT_DTYPE The names and datatype of the network input layers specified in the format [input_name datatype], for example: 'data' 'float32' Default is float32 if not specified Note that the quotes should always be included in order to handlespecial characters, spaces, etc. For multiple inputs specify multiple --input_dtype on the command line like: --input_dtype 'data1' 'float32' --input_dtype 'data2' 'float32' --input_encoding INPUT_ENCODING [INPUT_ENCODING ...], -e INPUT_ENCODING [INPUT_ENCODING ...] Usage: --input_encoding "INPUT_NAME" INPUT_ENCODING_IN [INPUT_ENCODING_OUT] Input encoding of the network inputs. Default is bgr. e.g. --input_encoding "data" rgba Quotes must wrap the input node name to handle special characters, spaces, etc. To specify encodings for multiple inputs, invoke --input_encoding for each one. e.g. --input_encoding "data1" rgba --input_encoding "data2" other Optionally, an output encoding may be specified for an input node by providing a second encoding. The default output encoding is bgr. e.g. --input_encoding "data3" rgba rgb Input encoding types: image color encodings: bgr,rgb, nv21, nv12, ... time_series: for inputs of rnn models; other: not available above or is unknown. Supported encodings: bgr rgb rgba argb32 nv21 nv12 time_series other --input_layout INPUT_NAME INPUT_LAYOUT, -l INPUT_NAME INPUT_LAYOUT Layout of each input tensor. If not specified, it will use the default based on the Source Framework, shape of input and input encoding. Accepted values are- NCDHW, NDHWC, NCHW, NHWC, NFC, NCF, NTF, TNF, NF, NC, F, NONTRIVIAL N = Batch, C = Channels, D = Depth, H = Height, W = Width, F = Feature, T = Time NDHWC/NCDHW used for 5d inputs NHWC/NCHW used for 4d image-like inputs NFC/NCF used for inputs to Conv1D or other 1D ops NTF/TNF used for inputs with time steps like the ones used for LSTM op NF used for 2D inputs, like the inputs to Dense/FullyConnected layers NC used for 2D inputs with 1 for batch and other for Channels (rarely used) F used for 1D inputs, e.g. Bias tensor NONTRIVIAL for everything elseFor multiple inputs specify multiple --input_layout on the command line. Eg: --input_layout "data1" NCHW --input_layout "data2" NCHW Note: This flag does not set the layout of the input tensor in the converted DLC. --custom_io CUSTOM_IO Use this option to specify a yaml file for custom IO. --preserve_io [PRESERVE_IO [PRESERVE_IO ...]] Use this option to preserve IO layout and datatype. The different ways of using this option are as follows: --preserve_io layout --preserve_io datatype In this case, user should also specify the string - layout or datatype in the command to indicate that converter needs to preserve the layout or datatype. e.g. --preserve_io layout input1 input2 output1 --preserve_io datatype input1 input2 output1 Optionally, the user may choose to preserve the layout and/or datatype for all the inputs and outputs of the graph. This can be done in the following two ways: --preserve_io layout --preserve_io datatype Additionally, the user may choose to preserve both layout and datatypes for all IO tensors by just passing the option as follows: --preserve_io Note: Only one of the above usages are allowed at a time. Note: --custom_io gets higher precedence than --preserve_io. --show_unconsumed_nodes Displays a list of unconsumed nodes, if there any are found. Nodes which are unconsumed do not violate the structural fidelity of the generated graph. --saved_model_tag SAVED_MODEL_TAG Specify the tag to seletet a MetaGraph from savedmodel. ex: --saved_model_tag serve. Default value will be 'serve' when it is not assigned. --saved_model_signature_key SAVED_MODEL_SIGNATURE_KEY Specify signature key to select input and output of the model. ex: --saved_model_signature_key serving_default. Default value will be 'serving_default' when it is not assigned --disable_batchnorm_folding --expand_lstm_op_structure Enables optimization that breaks the LSTM op to equivalent math ops --keep_disconnected_nodes Disable Optimization that removes Ops not connected to the main graph. This optimization uses output names provided over commandline OR inputs/outputs extracted from the Source model to determine the main graph -h, --help show this help message and exit --debug [DEBUG] Run the converter in debug mode. -o OUTPUT_PATH, --output_path OUTPUT_PATH Path where the converted Output model should be saved.If not specified, the converter model will be written to a file with same name as the input model --copyright_file COPYRIGHT_FILE Path to copyright file. If provided, the content of the file will be added to the output model. --float_bitwidth FLOAT_BITWIDTH Use the --float_bitwidth option to select the bitwidth to use when using float for parameters(weights/bias) and activations for all ops or specific Op (via encodings) selected through encoding, either 32 (default) or 16. --float_bw FLOAT_BW Note: --float_bw is deprecated, use --float_bitwidth. --float_bias_bw FLOAT_BIAS_BW Use the --float_bias_bw option to select the bitwidth to use for the float bias tensor --model_version MODEL_VERSION User-defined ASCII string to identify the model, only first 64 bytes will be stored --validation_target RUNTIME_TARGET PROCESSOR_TARGET A combination of processor and runtime target against which model will be validated. Choices for RUNTIME_TARGET: {cpu, gpu, dsp}. Choices for PROCESSOR_TARGET: {snapdragon_801, snapdragon_820, snapdragon_835}. If not specified, will validate model against {snapdragon_820, snapdragon_835} across all runtime targets. --strict If specified, will validate in strict mode whereby model will not be produced if it violates constraints of the specified validation target. If not specified, will validate model in permissive mode against the specified validation target. --udo_config_paths CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...], -udo CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...] Path to the UDO configs (space separated, if multiple) Quantizer Options: --quantization_overrides QUANTIZATION_OVERRIDES Use this option to specify a json file with parameters to use for quantization. These will override any quantization data carried from conversion (eg TF fake quantization) or calculated during the normal quantization process. Format defined as per AIMET specification. --keep_quant_nodes Use this option to keep activation quantization nodes in the graph rather than stripping them. Custom Op Package Options: --op_package_lib OP_PACKAGE_LIB, -opl OP_PACKAGE_LIB Use this argument to pass an op package library for quantization. Must be in the form and be separated by a comma for multiple package libs --converter_op_package_lib CONVERTER_OP_PACKAGE_LIB, -cpl CONVERTER_OP_PACKAGE_LIB Path to converter op package library compiled by the OpPackage generator. -p PACKAGE_NAME, --package_name PACKAGE_NAME A global package name to be used for each node in the Model.cpp file. Defaults to Qnn header defined package name --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...], -opc CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...] Path to a Qnn Op Package XML configuration file that contains user defined custom operations. Note: Only one of: {'package_name', 'op_package_config'} can be specified Copy to clipboard For more information, see [TensorFlow Model Conversion](https://docs.qualcomm.com/doc/80-63442-2/topic/model_conv_tensorflow.html) Additional details: - *input\_network argument:* > > > - The converter supports a single frozen graph .pb file, a > path to a pair of graph meta and checkpoint files, or the > path to a SavedModel directory (TF 2.x). > - If you are using the TensorFlow > Saver to save your graph during training, 3 files will be > generated as described below: > > > > > > > > > 1. <model-name>.meta > > 2. <model-name> > > 3. checkpoint > - The converter –input\_network option specifies the path > to the graph meta file. The converter will also use the > checkpoint file to read the graph nodes parameters during > conversion. The checkpoint file must have the same name > without the .meta suffix. > - This argument is required. - *input\_dim argument:* > > > - Specifies the input dimensions of the graph’s input > node(s) > - The converter requires a node name along with dimensions > as input from which it will create an input layer by > using the node output tensor dimensions. When defining a > graph, there is typically a placeholder name used as > input during training in the graph. The placeholder > tensor name is the name you must use as the argument. It > is also possible to use other types of nodes as input, > however the node used as input will not be used as part > of a layer other than the input layer. > - **Multiple Inputs** > > > > > > > > > - Networks with multiple inputs must provide > > –**input\_dim** INPUT\_NAME INPUT\_DIM, one for each > > input node. > > - This argument is required. - *out\_node argument:* > > > - The name of the last node in your TensorFlow graph which > will represent the output layer of your network. > - **Multiple Outputs** > > > > > > > > > - Networks with multiple outputs must provide several > > –**out\_node** arguments, one for each output node. - *output\_path argument:* > > > - Specifies the output DLC file name. > - This argument is optional. If not provided the converter > will create a DLC file with the same name as the graph > file name, with a .dlc file extension. - saved\_model\_tag: > > > - For Tensorflow 2.x networks, this option allows a > MetaGraph to be selected from the SavedModel specified by > input\_network. > - This argument is optional and defaults to “serve”. - saved\_model\_signature: > > > - For Tensorflow 2.x networks, this option specifies the > signature key for selecting inputs and outputs of a > Tensorflow 2.x SavedModel. > - This argument is optional and defaults to “serving\_default”. - SavedModel is the default model format in TensorFlow 2 and can been supported in Qualcomm® Neural Processing SDK TensorFlow Converter now. ### snpe-tflite-to-dlc snpe-tflite-to-dlc converts a TFLite model into a DLC file. usage: snpe-tflite-to-dlc [-d INPUT_NAME INPUT_DIM] [--signature_name SIGNATURE_NAME] [--out_node OUT_NAMES] [--input_type INPUT_NAME INPUT_TYPE] [--input_dtype INPUT_NAME INPUT_DTYPE] [--input_encoding ...] [--input_layout INPUT_NAME INPUT_LAYOUT] [--custom_io CUSTOM_IO] [--preserve_io [PRESERVE_IO [PRESERVE_IO ...]]] [--dump_relay DUMP_RELAY] [--quantization_overrides QUANTIZATION_OVERRIDES] [--keep_quant_nodes] [--disable_batchnorm_folding] [--expand_lstm_op_structure] [--keep_disconnected_nodes] --input_network INPUT_NETWORK [-h] [--debug [DEBUG]] [-o OUTPUT_PATH] [--copyright_file COPYRIGHT_FILE] [--float_bitwidth FLOAT_BITWIDTH] [--float_bw FLOAT_BW] [--float_bias_bw FLOAT_BIAS_BW] [--model_version MODEL_VERSION] [--validation_target RUNTIME_TARGET PROCESSOR_TARGET] [--strict] [--udo_config_paths CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] [--op_package_lib OP_PACKAGE_LIB] [--converter_op_package_lib CONVERTER_OP_PACKAGE_LIB] [-p PACKAGE_NAME | --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] Script to convert TFLite model into DLC required arguments: --input_network INPUT_NETWORK, -i INPUT_NETWORK Path to the source framework model. optional arguments: -d INPUT_NAME INPUT_DIM, --input_dim INPUT_NAME INPUT_DIM The names and dimensions of the network input layers specified in the format [input_name comma-separated-dimensions], for example: 'data' 1,224,224,3 Note that the quotes should always be included in order to handlespecial characters, spaces, etc. For multiple inputs specify multiple --input_dim on the command line like: --input_dim 'data1' 1,224,224,3 --input_dim 'data2' 1,50,100,3 --signature_name SIGNATURE_NAME, -sn SIGNATURE_NAME Specifies a specific subgraph signature to convert. --out_node OUT_NAMES, --out_name OUT_NAMES Name of the graph's output Tensor Names. Multiple output names should be provided separately like: --out_name out_1 --out_name out_2 --input_type INPUT_NAME INPUT_TYPE, -t INPUT_NAME INPUT_TYPE Type of data expected by each input op/layer. Type for each input is |default| if not specified. For example: "data" image.Note that the quotes should always be included in order to handle special characters, spaces,etc. For multiple inputs specify multiple --input_type on the command line. Eg: --input_type "data1" image --input_type "data2" opaque These options get used by DSP runtime and following descriptions state how input will be handled for each option. Image: Input is float between 0-255 and the input's mean is 0.0f and the input's max is 255.0f. We will cast the float to uint8ts and pass the uint8ts to the DSP. Default: Pass the input as floats to the dsp directly and the DSP will quantize it. Opaque: Assumes input is float because the consumer layer(i.e next layer) requires it as float, therefore it won't be quantized. Choices supported: image default opaque --input_dtype INPUT_NAME INPUT_DTYPE The names and datatype of the network input layers specified in the format [input_name datatype], for example: 'data' 'float32' Default is float32 if not specified Note that the quotes should always be included in order to handlespecial characters, spaces, etc. For multiple inputs specify multiple --input_dtype on the command line like: --input_dtype 'data1' 'float32' --input_dtype 'data2' 'float32' --input_encoding INPUT_ENCODING [INPUT_ENCODING ...], -e INPUT_ENCODING [INPUT_ENCODING ...] Usage: --input_encoding "INPUT_NAME" INPUT_ENCODING_IN [INPUT_ENCODING_OUT] Input encoding of the network inputs. Default is bgr. e.g. --input_encoding "data" rgba Quotes must wrap the input node name to handle special characters, spaces, etc. To specify encodings for multiple inputs, invoke --input_encoding for each one. e.g. --input_encoding "data1" rgba --input_encoding "data2" other Optionally, an output encoding may be specified for an input node by providing a second encoding. The default output encoding is bgr. e.g. --input_encoding "data3" rgba rgb Input encoding types: image color encodings: bgr,rgb, nv21, nv12, ... time_series: for inputs of rnn models; other: not available above or is unknown. Supported encodings: bgr rgb rgba argb32 nv21 nv12 time_series other --input_layout INPUT_NAME INPUT_LAYOUT, -l INPUT_NAME INPUT_LAYOUT Layout of each input tensor. If not specified, it will use the default based on the Source Framework, shape of input and input encoding. Accepted values are- NCDHW, NDHWC, NCHW, NHWC, NFC, NCF, NTF, TNF, NF, NC, F, NONTRIVIAL N = Batch, C = Channels, D = Depth, H = Height, W = Width, F = Feature, T = Time NDHWC/NCDHW used for 5d inputs NHWC/NCHW used for 4d image-like inputs NFC/NCF used for inputs to Conv1D or other 1D ops NTF/TNF used for inputs with time steps like the ones used for LSTM op NF used for 2D inputs, like the inputs to Dense/FullyConnected layers NC used for 2D inputs with 1 for batch and other for Channels (rarely used) F used for 1D inputs, e.g. Bias tensor NONTRIVIAL for everything elseFor multiple inputs specify multiple --input_layout on the command line. Eg: --input_layout "data1" NCHW --input_layout "data2" NCHW Note: This flag does not set the layout of the input tensor in the converted DLC. --custom_io CUSTOM_IO Use this option to specify a yaml file for custom IO. --preserve_io [PRESERVE_IO [PRESERVE_IO ...]] Use this option to preserve IO layout and datatype. The different ways of using this option are as follows: --preserve_io layout --preserve_io datatype In this case, user should also specify the string - layout or datatype in the command to indicate that converter needs to preserve the layout or datatype. e.g. --preserve_io layout input1 input2 output1 --preserve_io datatype input1 input2 output1 Optionally, the user may choose to preserve the layout and/or datatype for all the inputs and outputs of the graph. This can be done in the following two ways: --preserve_io layout --preserve_io datatype Additionally, the user may choose to preserve both layout and datatypes for all IO tensors by just passing the option as follows: --preserve_io Note: Only one of the above usages are allowed at a time. Note: --custom_io gets higher precedence than --preserve_io. --dump_relay DUMP_RELAY Dump Relay ASM and Params at the path provided with the argument Usage: --dump_relay --disable_batchnorm_folding --expand_lstm_op_structure Enables optimization that breaks the LSTM op to equivalent math ops --keep_disconnected_nodes Disable Optimization that removes Ops not connected to the main graph. This optimization uses output names provided over commandline OR inputs/outputs extracted from the Source model to determine the main graph -h, --help show this help message and exit --debug [DEBUG] Run the converter in debug mode. -o OUTPUT_PATH, --output_path OUTPUT_PATH Path where the converted Output model should be saved.If not specified, the converter model will be written to a file with same name as the input model --copyright_file COPYRIGHT_FILE Path to copyright file. If provided, the content of the file will be added to the output model. --float_bitwidth FLOAT_BITWIDTH Use the --float_bitwidth option to select the bitwidth to use when using float for parameters(weights/bias) and activations for all ops or specific Op (via encodings) selected through encoding, either 32 (default) or 16. --float_bw FLOAT_BW Note: --float_bw is deprecated, use --float_bitwidth. --float_bias_bw FLOAT_BIAS_BW Use the --float_bias_bw option to select the bitwidth to use for the float bias tensor --model_version MODEL_VERSION User-defined ASCII string to identify the model, only first 64 bytes will be stored --validation_target RUNTIME_TARGET PROCESSOR_TARGET A combination of processor and runtime target against which model will be validated. Choices for RUNTIME_TARGET: {cpu, gpu, dsp}. Choices for PROCESSOR_TARGET: {snapdragon_801, snapdragon_820, snapdragon_835}. If not specified, will validate model against {snapdragon_820, snapdragon_835} across all runtime targets. --strict If specified, will validate in strict mode whereby model will not be produced if it violates constraints of the specified validation target. If not specified, will validate model in permissive mode against the specified validation target. --udo_config_paths CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...], -udo CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...] Path to the UDO configs (space separated, if multiple) Quantizer Options: --quantization_overrides QUANTIZATION_OVERRIDES Use this option to specify a json file with parameters to use for quantization. These will override any quantization data carried from conversion (eg TF fake quantization) or calculated during the normal quantization process. Format defined as per AIMET specification. --keep_quant_nodes Use this option to keep activation quantization nodes in the graph rather than stripping them. Custom Op Package Options: --op_package_lib OP_PACKAGE_LIB, -opl OP_PACKAGE_LIB Use this argument to pass an op package library for quantization. Must be in the form and be separated by a comma for multiple package libs --converter_op_package_lib CONVERTER_OP_PACKAGE_LIB, -cpl CONVERTER_OP_PACKAGE_LIB Path to converter op package library compiled by the OpPackage generator. -p PACKAGE_NAME, --package_name PACKAGE_NAME A global package name to be used for each node in the Model.cpp file. Defaults to Qnn header defined package name --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...], -opc CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...] Path to a Qnn Op Package XML configuration file that contains user defined custom operations. Note: Only one of: {'package_name', 'op_package_config'} can be specified Copy to clipboard For more information, see [TFLite Model Conversion](https://docs.qualcomm.com/doc/80-63442-2/topic/model_conv_tflite.html) Additional details: - *input\_network argument:* > > > - The converter supports a single .tflite file. > - The converter –input\_network option specifies the path > to the .tflite file. > - This argument is required. - *input\_dim argument:* > > > - Specifies the input dimensions of the graph’s input > node(s) > - The converter requires a node name along with dimensions > as input from which it will create an input layer by > using the node output tensor dimensions. When defining a > graph, there is typically a placeholder name used as > input during training in the graph. The placeholder > tensor name is the name you must use as the argument. It > is also possible to use other types of nodes as input, > however the node used as input will not be used as part > of a layer other than the input layer. > - **Multiple Inputs** > > > > > > > > > - Networks with multiple inputs must provide > > –**input\_dim** INPUT\_NAME INPUT\_DIM, one for each > > input node. > - This argument is optional. - *output\_path argument:* > > > - Specifies the output DLC file name. > - This argument is optional. If not provided the converter > will create a DLC file with the same name as the tflite > file name, with a .dlc file extension. ### qairt-converter The **qairt-converter** tool converts a model from the one of Onnx/TensorFlow/TFLite/PyTorch framework to a DLC file representing the QNN graph format that can enable inference on Qualcomm AI IP/HW. The converter auto detects the framework based on the source model extension. Current ONNX Conversion supports upto ONNX Opset 22. Basic command line usage looks like: usage: qairt-converter [--source_model_input_shape INPUT_NAME INPUT_DIM] [--out_tensor_node OUT_NAMES] [--source_model_input_datatype INPUT_NAME INPUT_DTYPE] [--source_model_input_layout INPUT_NAME INPUT_LAYOUT] [--desired_input_layout INPUT_NAME DESIRED_INPUT_LAYOUT] [--source_model_output_layout OUTPUT_NAME OUTPUT_LAYOUT] [--desired_output_layout OUTPUT_NAME DESIRED_OUTPUT_LAYOUT] [--desired_input_color_encoding [ ...]] [--preserve_io_datatype [PRESERVE_IO_DATATYPE ...]] [--dump_config_template DUMP_IO_CONFIG_TEMPLATE] [--config IO_CONFIG] [--dry_run [DRY_RUN]] [--enable_framework_trace] [--remove_unused_inputs] [--gguf_config GGUF_CONFIG] [--quantizer_log QUANTIZER_LOG] [--quantizer_log_level {LogLevel.NONE,LogLevel.TRACE,LogLevel.INFO}] [--quantization_overrides QUANTIZATION_OVERRIDES] [--lora_weight_list LORA_WEIGHT_LIST] [--quant_updatable_mode {none,adapter_only,all}] [--onnx_skip_simplification] [--onnx_override_batch BATCH] [--onnx_define_symbol SYMBOL_NAME VALUE] [--onnx_validate_models] [--onnx_summary] [--onnx_perform_sequence_construct_optimizer] [--tf_summary] [--tf_override_batch BATCH] [--tf_disable_optimization] [--tf_show_unconsumed_nodes] [--tf_saved_model_tag SAVED_MODEL_TAG] [--tf_saved_model_signature_key SAVED_MODEL_SIGNATURE_KEY] [--tf_validate_models] [--tflite_signature_name SIGNATURE_NAME] [--dump_exported_onnx] --input_network INPUT_NETWORK [--debug [DEBUG]] [--output_path OUTPUT_PATH] [--copyright_file COPYRIGHT_FILE] [--float_bitwidth FLOAT_BITWIDTH] [--float_bias_bitwidth FLOAT_BIAS_BITWIDTH] [--set_model_version MODEL_VERSION] [--export_format EXPORT_FORMAT] [--converter_op_package_lib CONVERTER_OP_PACKAGE_LIB] [--package_name PACKAGE_NAME | --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...]] [--target_backend BACKEND] [--target_soc_model SOC_MODEL] [-h] required arguments: --input_network INPUT_NETWORK, -i INPUT_NETWORK Path to the source framework model. optional arguments: --source_model_input_shape INPUT_NAME INPUT_DIM, -s INPUT_NAME INPUT_DIM The name and dimension of all the input buffers to the network specified in the format [input_name comma-separated-dimensions], for example: --source_model_input_shape 'data' 1,224,224,3. Note that the quotes should always be included in order to handle special characters, spaces, etc. For scalar inputs, use a single dimension `0` to indicate that the input is a scalar value. This representation is supported for ONNX models only. For multiple inputs specify multiple --source_model_input_shape on the commandline like: --source_model_input_shape 'data1' 1,224,224,3 --source_model_input_shape 'data2' 0 NOTE: Required for TensorFlow and PyTorch. Optional for Onnx and Tflite In case of Onnx, this feature works only with Onnx 1.6.0 and above --out_tensor_node OUT_NAMES, --out_tensor_name OUT_NAMES Name of the graph's output Tensor Names. Multiple output names should be provided separately like: --out_tensor_name out_1 --out_tensor_name out_2 NOTE: Required for TensorFlow. Optional for Onnx, Tflite and PyTorch --source_model_input_datatype INPUT_NAME INPUT_DTYPE The names and datatype of the network input layers specified in the format [input_name datatype], for example: 'data' 'float32' Default is float32 if not specified Note that the quotes should always be included in order to handlespecial characters, spaces, etc. For multiple inputs specify multiple --source_model_input_datatype on the command line like: --source_model_input_datatype 'data1' 'float32' --source_model_input_datatype 'data2' 'float32' --source_model_input_layout INPUT_NAME INPUT_LAYOUT Layout of each input tensor. If not specified, it will use the default based on the Source Framework, shape of input and input encoding. Accepted values are- NCDHW, NDHWC, NCHW, NHWC, HWIO, OIHW, NFC, NCF, NTF, TNF, NF, NC, F N = Batch, C = Channels, D = Depth, H = Height, W = Width, F = Feature, T = Time, I = Input, O = Output NDHWC/NCDHW used for 5d inputs NHWC/NCHW used for 4d image-like inputs HWIO/IOHW used for Weights of Conv Ops NFC/NCF used for inputs to Conv1D or other 1D ops NTF/TNF used for inputs with time steps like the ones used for LSTM op NF used for 2D inputs, like the inputs to Dense/FullyConnected layers NC used for 2D inputs with 1 for batch and other for Channels (rarely used) F used for 1D inputs, e.g. Bias tensor For multiple inputs specify multiple --source_model_input_layout on the command line. Eg: --source_model_input_layout "data1" NCHW --source_model_input_layout "data2" NCHW --desired_input_layout INPUT_NAME DESIRED_INPUT_LAYOUT Desired Layout of each input tensor. If not specified, it will use the default based on the Source Framework, shape of input and input encoding. Accepted values are- NCDHW, NDHWC, NCHW, NHWC, HWIO, OIHW, NFC, NCF, NTF, TNF, NF, NC, F N = Batch, C = Channels, D = Depth, H = Height, W = Width, F = Feature, T = Time, I = Input, O = Output NDHWC/NCDHW used for 5d inputs NHWC/NCHW used for 4d image-like inputs HWIO/IOHW used for Weights of Conv Ops NFC/NCF used for inputs to Conv1D or other 1D ops NTF/TNF used for inputs with time steps like the ones used for LSTM op NF used for 2D inputs, like the inputs to Dense/FullyConnected layers NC used for 2D inputs with 1 for batch and other for Channels (rarely used) F used for 1D inputs, e.g. Bias tensor For multiple inputs specify multiple --desired_input_layout on the command line. Eg: --desired_input_layout "data1" NCHW --desired_input_layout "data2" NCHW --source_model_output_layout OUTPUT_NAME OUTPUT_LAYOUT Layout of each output tensor. If not specified, it will use the default based on the Source Framework, shape of input and input encoding. Accepted values are- NCDHW, NDHWC, NCHW, NHWC, HWIO, OIHW, NFC, NCF, NTF, TNF, NF, NC, F N = Batch, C = Channels, D = Depth, H = Height, W = Width, F = Feature, T = Time NDHWC/NCDHW used for 5d inputs NHWC/NCHW used for 4d image-like inputs NFC/NCF used for inputs to Conv1D or other 1D ops NTF/TNF used for inputs with time steps like the ones used for LSTM op NF used for 2D inputs, like the inputs to Dense/FullyConnected layers NC used for 2D inputs with 1 for batch and other for Channels (rarely used) F used for 1D inputs, e.g. Bias tensor For multiple inputs specify multiple --source_model_output_layout on the command line. Eg: --source_model_output_layout "data1" NCHW --source_model_output_layout "data2" NCHW --desired_output_layout OUTPUT_NAME DESIRED_OUTPUT_LAYOUT Desired Layout of each output tensor. If not specified, it will use the default based on the Source Framework. Accepted values are- NCDHW, NDHWC, NCHW, NHWC, HWIO, OIHW, NFC, NCF, NTF, TNF, NF, NC, F N = Batch, C = Channels, D = Depth, H = Height, W = Width, F = Feature, T = Time NDHWC/NCDHW used for 5d outputs NHWC/NCHW used for 4d image-like outputs NFC/NCF used for outputs to Conv1D or other 1D ops NTF/TNF used for outputs with time steps like the ones used for LSTM op NF used for 2D outputs, like the outputs to Dense/FullyConnected layers NC used for 2D outputs with 1 for batch and other for Channels (rarely used) F used for 1D outputs, e.g. Bias tensor For multiple outputs specify multiple --desired_output_layout on the command line. Eg: --desired_output_layout "data1" NCHW --desired_output_layout "data2" NCHW --desired_input_color_encoding [ ...], -e [ ...] Usage: --input_color_encoding "INPUT_NAME" INPUT_ENCODING_IN [INPUT_ENCODING_OUT] Input encoding of the network inputs. Default is bgr. e.g. --input_color_encoding "data" rgba Quotes must wrap the input node name to handle special characters, spaces, etc. To specify encodings for multiple inputs, invoke --input_color_encoding for each one. e.g. --input_color_encoding "data1" rgba --input_color_encoding "data2" other Optionally, an output encoding may be specified for an input node by providing a second encoding. The default output encoding is bgr. e.g. --input_color_encoding "data3" rgba rgb Input encoding types: image color encodings: bgr,rgb, nv21, nv12, ... time_series: for inputs of rnn models; other: not available above or is unknown. Supported encodings: bgr rgb rgba argb32 nv21 nv12 --preserve_io_datatype [PRESERVE_IO_DATATYPE ...] Use this option to preserve IO datatype. The different ways of using this option are as follows: --preserve_io_datatype e.g. --preserve_io_datatype input1 input2 output1 The user may choose to preserve the datatype for all the inputs and outputs of the graph. --preserve_io_datatype Note: --config gets higher precedence than --preserve_io_datatype. --dump_config_template DUMP_IO_CONFIG_TEMPLATE Dumps the yaml template for I/O configuration. This file can be edited as per the custom requirements and passed using the option --configUse this option to specify a yaml file to which the IO config template is dumped. --config IO_CONFIG Use this option to specify a yaml file for input and output options. --dry_run [DRY_RUN] Evaluates the model without actually converting any ops, and returns unsupported ops/attributes as well as unused inputs and/or outputs if any. --enable_framework_trace Use this option to enable converter to trace the op/tensor change information. Currently framework op trace is supported only for ONNX converter. --remove_unused_inputs Use this option to remove the disconnected graph input nodes after the conversion --gguf_config GGUF_CONFIG This is an optional argument that can be used when input network is a GGUF File.It specifies the path to the config file for building GenAI model.(the config.json file generated when saving the huggingface model) --quantizer_log QUANTIZER_LOG Valid for use with v2.0.0 JSON schema for quantization overrides or when --use_quantize_v2 is provided. Enable logging in the quantizer, logging to the file . E.g., --quantizer_log my_model_name.csv will produce the file my_model_name.csv. See --quantizer_log_level. --quantizer_log_level {LogLevel.NONE,LogLevel.TRACE,LogLevel.INFO} Sets the logging level in the quantizer. See --quantizer_log. INFO: Emits a file in the CSV format. Requires --quantizer_log to be set. Warnings and errors are emitted to the console. TRACE: Emits a file in the TXT format. Requires --quantizer_log to be set. Warnings and errors are emitted to the console. NONE: Default value. No file is emitted. Warnings and errors are emitted to the console. --debug [DEBUG] Run the converter in debug mode. --output_path OUTPUT_PATH, -o OUTPUT_PATH Path where the converted Output model should be saved.If not specified, the converter model will be written to a file with same name as the input model --copyright_file COPYRIGHT_FILE Path to copyright file. If provided, the content of the file will be added to the output model. --float_bitwidth FLOAT_BITWIDTH Use the --float_bitwidth option to convert the graph to the specified float bitwidth, either 32 (default) or 16. --float_bias_bitwidth FLOAT_BIAS_BITWIDTH Use the --float_bias_bitwidth option to select the bitwidth to use for float bias tensor, either 32 or 16 (default '0' if not provided). --set_model_version MODEL_VERSION User-defined ASCII string to identify the model, only first 64 bytes will be stored --export_format EXPORT_FORMAT DLC_DEFAULT (default) - Produce a Float graph given a Float Source graph - Produce a Quant graph given a Source graph with provided Encodings DLC_STRIP_QUANT - Produce a Float Graph with discarding Quant data -h, --help show this help message and exit Custom Op Package Options: --converter_op_package_lib CONVERTER_OP_PACKAGE_LIB, -cpl CONVERTER_OP_PACKAGE_LIB Absolute path to converter op package library compiled by the OpPackage generator. Must be separated by a comma for multiple package libraries. Note: Order of converter op package libraries must follow the order of xmls. Ex1: --converter_op_package_lib absolute_path_to/libExample.so Ex2: -cpl absolute_path_to/libExample1.so,absolute_path_to/libExample2.so --package_name PACKAGE_NAME, -p PACKAGE_NAME A global package name to be used for each node in the Model.cpp file. Defaults to Qnn header defined package name --op_package_config CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...], -opc CUSTOM_OP_CONFIG_PATHS [CUSTOM_OP_CONFIG_PATHS ...] Path to a Qnn Op Package XML configuration file that contains user defined custom operations. Quantizer Options: --quantization_overrides QUANTIZATION_OVERRIDES, -q QUANTIZATION_OVERRIDES Use this option to specify a json file with parameters to use for quantization. These will override any quantization data carried from conversion (eg TF fake quantization) or calculated during the normal quantization process. Format defined as per AIMET specification. LoRA Converter Options: --lora_weight_list LORA_WEIGHT_LIST Path to a file specifying a list of tensor names that should be updateable. --quant_updatable_mode {none,adapter_only,all} Specify whether/for which tensors the quantization encodings change across use-cases. In none mode, no quantization encodings are updatable. In adapter_only mode quantization encodings for only lora/adapter branch (Conv->Mul->Conv) change across use-case, the base branch quantization encodings remain the same. In all mode, all quantization encodings are updatable. Onnx Converter Options: --onnx_skip_simplification, -oss Do not attempt to simplify the model automatically. This may prevent some models from properly converting when sequences of unsupported static operations are present. --onnx_override_batch BATCH The batch dimension override. This will take the first dimension of all inputs and treat it as a batch dim, overriding it with the value provided here. For example: --onnx_override_batch 6 will result in a shape change from [1,3,224,224] to [6,3,224,224]. If there are inputs without batch dim this should not be used and each input should be overridden independently using -s option for input dimension overrides. --onnx_define_symbol SYMBOL_NAME VALUE This option allows overriding specific input dimension symbols. For instance you might see input shapes specified with variables such as : data: [1,3,height,width] To override these simply pass the option as: --onnx_define_symbol height 224 --onnx_define_symbol width 448 which results in dimensions that look like: data: [1,3,224,448] --onnx_validate_models Validate the original ONNX model against optimized ONNX model. Constant inputs with all value 1s will be generated and will be used by both models and their outputs are checked against each other. The % average error and 90th percentile of output differences will be calculated for this. Note: Usage of this flag will incur extra time due to inference of the models. --onnx_summary Summarize the original onnx model and optimized onnx model. Summary will print the model information such as number of parameters, number of operators and their count, input-output tensor name, shape and dtypes. --onnx_perform_sequence_construct_optimizer This option allows optimization on SequenceConstruct Op. When SequenceConstruct op is one of the outputs of the graph, it removes SequenceConstruct op and makes its inputs as graph outputs to replace the original output of SequenceConstruct. --tf_summary Summarize the original TF model and optimized TF model. Summary will print the model information such as number of parameters, number of operators and their count, input-output tensor name, shape and dtypes. TensorFlow Converter Options: --tf_override_batch BATCH The batch dimension override. This will take the first dimension of all inputs and treat it as a batch dim, overriding it with the value provided here. For example: --tf_override_batch 6 will result in a shape change from [1,224,224,3] to [6,224,224,3]. If there are inputs without batch dim this should not be used and each input should be overridden independently using -s option for input dimension overrides. --tf_disable_optimization Do not attempt to optimize the model automatically. --tf_show_unconsumed_nodes Displays a list of unconsumed nodes, if there any are found. Nodeswhich are unconsumed do not violate the structural fidelity of thegenerated graph. --tf_saved_model_tag SAVED_MODEL_TAG Specify the tag to seletet a MetaGraph from savedmodel. ex: --saved_model_tag serve. Default value will be 'serve' when it is not assigned. --tf_saved_model_signature_key SAVED_MODEL_SIGNATURE_KEY Specify signature key to select input and output of the model. ex: --tf_saved_model_signature_key serving_default. Default value will be 'serving_default' when it is not assigned --tf_validate_models Validate the original TF model against optimized TF model. Constant inputs with all value 1s will be generated and will be used by both models and their outputs are checked against each other. The % average error and 90th percentile of output differences will be calculated for this. Note: Usage of this flag will incur extra time due to inference of the models. Tflite Converter Options: --tflite_signature_name SIGNATURE_NAME Use this option to specify a specific Subgraph signature to convert PyTorch Converter Options: --dump_exported_onnx Dump the exported Onnx model from input Torchscript model Backend Options: --target_backend BACKEND Use this option to specify the backend on which the model needs to run. Providing this option will generate a graph optimized for the given backend and this graph may not run on other backends. The default backend is HTP. Supported backends are CPU,GPU,DSP,HTP,HTA,LPAI. --target_soc_model SOC_MODEL Use this option to specify the SOC on which the model needs to run. This can be found from SOC info of the device and it starts with strings such as SDM, SM, QCS, IPQ, SA, QC, SC, SXR, SSG, STP, QRB, or AIC. NOTE: --target_backend option must be provided to use --target_soc_model option. Note: Only one of: {'package_name', 'op_package_config'} can be specified Copy to clipboard ## Model Preparation ### snpe-dlc-graph-prepare snpe-dlc-graph-prepare is used to perform offline graph preparation on quantized dlcs to run on DSP/HTP runtimes. Command Line Options: [ -h, --help ] Displays this help message. [ --version ] Displays version information. [ --verbose ] Enable verbose user messages. [ --quiet ] Disables some user messages. [ --silent ] Disables all but fatal user messages. [ --debug= ] Sets the debug log level. [ --debug1 ] Enables level 1 debug messages. [ --debug2 ] Enables level 2 debug messages. [ --debug3 ] Enables level 3 debug messages. [ --log-mask= ] Sets the debug log mask to set the log level for one or more areas. Example: ".*=USER_ERROR, .*=INFO, NDK=DEBUG2, NCC=DEBUG3" [ --log-file= ] Overrides the default name for the debug log file. [ --log-dir= ] Overrides the default directory path where debug log files are written. [ --log-file-include-hostname ] Appends the name of this host to the log file name. --input_dlc= Path to the dlc container containing the model for which graph cache should be generated. This argument is required. [ --output_dlc= ] Path at which the cached data included model container should be written. If this argument is omitted, the quantized model will be written at _cached.dlc. [ --set_output_tensors= ] Specifies a comma separated list of tensors to be output after execution without whitespace. [ --set_output_layers= ] Specifies a comma separated list of layers whose output buffers should be output after execution, without whitespace. [ --input_list= ] Path to a file specifying input images as passed to snpe-net-run. Only the graph output buffers information specified in the input list (line starting with # or %, if any) will be used. Paths to the input images will be ignored [ --htp_socs= ] Specify SoC(s) to generate HTP Offline Cache for. SoCs are specified with an ASIC identifier, in a comma seperated list without whitespace. For example --htp_socs sm8350,sm8450,sm8550,sm8650,qcs6490,qcs8550. This flag and --htp_archs are mutually exclusive. Default ASIC identifier: sm8650 [ --htp_archs= ] Specify DSP Architecture(s) to generate general HTP Offline Cache for. Architectures are specified with an ASIC identifier, in a comma seperated list without whitespace. For example, --htp_archs v68,v73. This flag cannot be coupled with --htp_socs or --vtcm_override [ --vtcm_override= ] Specify a single value representing the VTCM size in MB for the generated HTP Offline Caches. For example, --vtcm_override 4. When set to 0, the SoC maximum VTCM size is used and if cache compatibility mode is set to STRICT the maximum value is checked. This flag can be used with --htp_socs to override the default SOC VTCM size setting [ --optimization_level= ] Specify an optimization level. Valid values are 1, 2 and 3. Default is 2. Higher optimization levels incur longer offline prepare time but yield more optimal graph and hence faster execution time for most graphs [ --optimization_preset== ] Specify an optimization preset. Valid values are any integer value greater than or equal to zero. Default is 0. These are experimental HTP graph compiler settings that typically affect latency and DRAM bandwidth. These presets are intended for use with optimization_level=3. Unlike optimization levels, preset values do not follow a consistent performance pattern. Results may vary depending on the network architecture and software release. [ --buffer_data_type= ] Sets data type of IO buffers during prepare. Data Type can be the following: float32, fixedPoint8, fixedPoint16. Arguments should be formatted as follows: --buffer_data_type buffer_name1=buffer_name1_data_type --buffer_data_type buffer_name2=buffer_name2_data_type (Note: deprecated) [ --overwrite_cache_records ] Allow this tool to overwrite over any cache record that exactly matches the requested SoC(s). Default behavior is to skip (re)generating cache records when a matching cache already exists [ --use_float_io ] Prepare quantized HTP Graph to operate with floating point inputs/outputs (Note: deprecated) [ --htp_dlbc= ] Specify Deep Learning Bandwidth Compression (DLBC) for this HTP graph. The default setting is OFF. To turn on, specify it as --htp_dlbc=true [ --num_hvx_threads= ] Specify the number of HVX threads to reserve for this HTP graph. Must be greater than 0. [ --input_name= ] Specifies the name of input for which dimensions are specified e.g. --input_name= If the DLC has multiple graphs, graph names are required. Quotes are required if graph names are specified. Graph names can be found in snpe-dlc-info. e.g. --input_name=" " [ --input_dimensions= ] Specifies new dimensions for input whose name is specified in input_name. e.g. --input_dimension=1,224,224,3 For multiple inputs, specify --input_name= and --input_dimensions= multiple times. If the DLC has multiple graphs, graph names are required. Quotes are required if graph names are specified. Graph names can be found in snpe-dlc-info. e.g. --input_dimensions=" 1,224,224,3" [ --memorymapped_buffer_hint= ] Specifies memory-mapped buffers hint. The default setting is OFF. To turn on, specify it as --memorymapped_buffer_hint=true [ --udo_package_path= ] Use this option to specify path to the Registration Library for UDO Package(s). Usage is: --udo_package_path= Optionally, user can provide multiple packages as a comma-separated list. This option must be specified for Networks with UDO. All UDO's in Network must have host executable CPU Implementation Copy to clipboard For detailed information on how to use the tool, please refer to [Offline Graph Caching for DSP Runtime on HTP](https://docs.qualcomm.com/doc/80-63442-2/topic/offline_graph_caching.html) ### snpe-dlc-quant snpe-dlc-quant converts non-quantized DLC models into quantized DLC models. Command Line Options: [ -h,--help ] Displays this help message. [ --version ] Displays version information. [ --verbose ] Enable verbose user messages. [ --quiet ] Disables some user messages. [ --silent ] Disables all but fatal user messages. [ --debug= ] Sets the debug log level. [ --debug1 ] Enables level 1 debug messages. [ --debug2 ] Enables level 2 debug messages. [ --debug3 ] Enables level 3 debug messages. [ --log-mask= ] Sets the debug log mask to set the log level for one or more areas. Example: ".*=USER_ERROR, .*=INFO, NDK=DEBUG2, NCC=DEBUG3" [ --log-file= ] Overrides the default name for the debug log file. [ --log-dir= ] Overrides the default directory path where debug log files are written. [ --log-file-include-hostname ] Appends the name of this host to the log file name. [ --input_dlc= ] Path to the dlc container containing the model for which fixed-point encoding metadata should be generated. This argument is required. [ --input_list= ] Path to a file specifying the trial inputs. This file should be a plain text file, containing one or more absolute file paths per line. These files will be taken to constitute the trial set. Each path is expected to point to a binary file containing one trial input in the 'raw' format, ready to be consumed by the tool without any further modifications. This is similar to how input is provided to snpe-net-run application. [ --no_weight_quantization ] Note: Deprecated. [ --output_dlc= ] Path at which the metadata-included quantized model container should be written. If this argument is omitted, the quantized model will be written at _quantized.dlc. [ --use_enhanced_quantizer ] Note: Deprecated; use --param_quantizer and/or --act_quantizer. Use the enhanced quantizer feature when quantizing the model. Regular quantization determines the range using the actual values of min and max of the data being quantized. Enhanced quantization uses an algorithm to determine optimal range. It can be useful for quantizing models that have long tails in the distribution of the data being quantized. [ --use_adjusted_weights_quantizer ] Note: Deprecated; use --param_quantizer. Use the adjusted tf quantizer for quantizing the weights only. This might be helpful for improving the accuracy of some models, such as denoise model as being tested. This option is only used when quantizing the weights with 8 bit. [ --optimizations= ] Note: Deprecated; use --algorithms. Enables new optimization algorithms. Usage is: --optimizations --optimizations Available optimization algorithms are: "cle" - Cross layer equalization includes a number of methods for equalizing weights and biases across layers in order to rectify imbalances that cause quantization errors. [ --algorithms= ] Enables new optimization algorithms. Usage is: --algorithms --algorithms Available optimization algorithms are: "cle" - Cross layer equalization includes a number of methods for equalizing weights and biases across layers in order to rectify imbalances that cause quantization errors. [ --override_params ] Use this option to override quantization parameters when quantization was provided from the original source framework (eg TF fake quantization). Note: Quantizer throws an error if overridden encodings contain unsupported bitwidths. [ --use_encoding_optimizations ] Note: Deprecated. [ --udo_package_path= ] Specifies the path to the registration library for UDO package(s). Usage is: --udo_package_path= You can (optionally) provide multiple packages as a comma-separated list. This option must be specified for networks with UDO. All UDO's in a network must have a host-executable CPU implementation. [ --use_symmetric_quantize_weights ] Note: Deprecated, use --param_quantizer. Use the symmetric quantizer feature when quantizing the weights of the model. It makes sure min and max have the same absolute values about zero. Symmetrically quantized data will also be stored as int#_t data such that the offset is always 0. [ --use_native_dtype ] Note: This option is deprecated, use --use_native_input_files option in future. Use this option to indicate how to read input files, 1. float (default): reads inputs as floats and quantizes if necessary based on quantization parameters in the model. 2. native: reads inputs assuming the data type to be native to the model. For ex., uint8_t. [ --use_native_input_files ] Use this option to indicate how to read input files, 1. float (default): reads inputs as floats and quantizes if necessary based on quantization parameters in the model. 2. native: reads inputs assuming the data type to be native to the model. For ex., uint8_t. [ --use_native_output_files ] Use this option to indicate the data type of the output files, 1. float (default): generates the output file as float data. 2. native: generates the output file as datatype native to the source model. i.e. uint8_t. [ --float_fallback ] Enables fallback to floating point (FP) instead of fixed point. This option can be paired with --float_bitwidth to indicate the bitwidth for FP (by default 32). If this option is enabled, then input list must not be provided and --override_params must be provided. The external quantization file (encoding file) might be missing quantization parameters for some interim tensors. First it will try to fill the gaps by propagating across math-invariant functions. If the quantization parameters are still missing, it applies fallback to nodes to floating point [ --param_quantizer= ] Indicates the weight/bias quantizer to use. Optional and must be followed by one of the following options: "tf": Uses the real min/max of the data and specified bitwidth (default). "enhanced": Uses an algorithm useful for quantizing models with long tails present in the weight distribution. "symmetric": Ensures min and max have the same absolute values about zero. Data will be stored as int#_t data such that the offset is always 0. [ --act_quantizer= ] Indicates the activation quantizer to use. Optional and must be followed by one of the following options: "tf": Uses the real min/max of the data and specified bitwidth (default). "enhanced": Uses an algorithm useful for quantizing models with long tails present in the weight distribution. "symmetric": Ensures min and max have the same absolute values about zero. Data will be stored as int#_t data such that the offset is always 0. [ --bitwidth= ] Note: Deprecated. Selects the bitwidth to use when quantizing the weights/activations/biases; 8 (default) or 16. Cannot be mixed with --weights_bitwidth or --act_bitwidth or --bias_bitwidth. [ --weights_bitwidth= ] Selects the bitwidth to use when quantizing the weights; either 4, 8 (default) or 16. 8w/16a is only supported by HTA currently. Cannot be mixed with --bitwidth. [ --act_bitwidth= ] Selects the bitwidth to use when quantizing the activations; either 8 (default) or 16. 8w/16a is only supported by HTA currently. Cannot be mixed with --bitwidth. [ --float_bitwidth= ] Selects the bitwidth to use when using float for parameters (weights/biases) and activations for all ops or a specific op (via encodings) selected through encoding; either 32 (default) or 16. [ --bias_bitwidth= ] Selects the bitwidth to use when quantizing the biases; either 8 (default) or 32. Using 32-bit biases may sometimes provide a small improvement in accuracy. Cannot be mixed with --bitwidth. [ --float_bias_bitwidth= ] Specifies the bitwidth for float bias tensors; either 32 or 16. If not provided and bias is overridden to float in the quantizer, the overriding float tensor's bitwidth will be used. [ --axis_quant ] Note: Deprecated; use --use_per_channel_quantization. Selects per-axis-element quantization for the weights and biases of certain layer types. Only Convolution, Deconvolution, and FullyConnected are supported. [ --use_per_channel_quantization ] Selects per-axis-element quantization for the weights and biases of certain layer types. Only Convolution, Deconvolution, and FullyConnected are supported. [ --use_per_row_quantization ] Enables row wise quantization of Matmul and FullyConnected ops. [ --enable_per_row_quantized_bias ] Enables row wise quantization of bias for FullyConnected op, when weights are per-row quantized. [ --restrict_quantization_steps= ] Specifies the number of steps to use for computing quantization encodings such that scale = (max - min) / number of quantization steps. The option should be passed as a comma separated pair of hexadecimal string minimum and maximum values, i.e., --restrict_quantization_steps "MIN,MAX". Note that this is a hexadecimal string literal and not a signed integer, to supply a negative value an explicit minus sign is required, e.g., --restrict_quantization_steps "-0x80,0x7F" indicates an example 8-bit range. --restrict_quantization_steps "-0x8000,0x7F7F" indicates an example 16-bit range. This option only applies to symmetric parameter quantization. Description: Generate 8 or 16 bit TensorFlow style fixed point weight and activations encodings for a floating point DLC. Copy to clipboard Additional details: - For specifying input\_list, refer to input\_list argument in [snpe-net-run](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-net-run) for supported input formats (in order to calculate output activation encoding information for all layers, **do not** include the line which specifies desired outputs). - The tool requires the batch dimension of the DLC input file to be set to 1 during the original model conversion step. > > > - An example of quantization using snpe-dlc-quant can be found > in the C/C++ Tutorial section: [Running the Inception v3 > Model](https://docs.qualcomm.com/doc/80-63442-2/topic/tutorial_inceptionv3.html). For details on > quantization see [Quantized vs Non-Quantized > Models](https://docs.qualcomm.com/doc/80-63442-2/topic/quantized_models.html). > - Outputs can be specified for snpe-dlc-quant by modifying the > input\_list in the following ways: > > > > > > > > > #[] > > %[] > > :=[:=] > > … > > Copy to clipboard > > > > > > **Note:** Output tensors and layers can be specified > > individually, but when specifying both, the order shown must > > be used to specify each. > - When using the Qualcomm® Neural Processing SDK API: > > > > > > > > > - Any output layers specified when snpe-dlc-quant was > > called, need to be specified using the > > Snpe\_SNPEBuilder\_SetOutputLayers() > > function. > > - Any output tensors specified when snpe-dlc-quant was > > called, need to be specified using the > > Snpe\_SNPEBuilder\_SetOutputTensors() > > function. ### snpe-dlc-quantize snpe-dlc-quantize converts non-quantized DLC models into quantized DLC models. Command Line Options: [ -h,--help ] Displays this help message. [ --version ] Displays version information. [ --verbose ] Enable verbose user messages. [ --quiet ] Disables some user messages. [ --silent ] Disables all but fatal user messages. [ --debug= ] Sets the debug log level. [ --debug1 ] Enables level 1 debug messages. [ --debug2 ] Enables level 2 debug messages. [ --debug3 ] Enables level 3 debug messages. [ --log-mask= ] Sets the debug log mask to set the log level for one or more areas. Example: ".*=USER_ERROR, .*=INFO, NDK=DEBUG2, NCC=DEBUG3" [ --log-file= ] Overrides the default name for the debug log file. [ --log-dir= ] Overrides the default directory path where debug log files are written. [ --log-file-include-hostname ] Appends the name of this host to the log file name. [ --input_dlc= ] Path to the dlc container containing the model for which fixed-point encoding metadata should be generated. This argument is required. [ --input_list= ] Path to a file specifying the trial inputs. This file should be a plain text file, containing one or more absolute file paths per line. These files will be taken to constitute the trial set. Each path is expected to point to a binary file containing one trial input in the 'raw' format, ready to be consumed by the tool without any further modifications. This is similar to how input is provided to snpe-net-run application. [ --no_weight_quantization ] Note: Deprecated. [ --output_dlc= ] Path at which the metadata-included quantized model container should be written. If this argument is omitted, the quantized model will be written at _quantized.dlc. [ --enable_htp ] Pack HTP information in quantized DLC. [ --htp_socs= ] Specify SoC to generate HTP Offline Cache for. SoCs are specified with an ASIC identifier, in a comma separated list. For example, --htp_socs sm8650 [ --overwrite_cache_records ] Overwrite HTP cache records present in the DLC. [ --use_float_io ] Pack HTP information in quantized DLC (Note: deprecated). [ --use_enhanced_quantizer ] Note: Deprecated; use --param_quantizer and/or --act_quantizer. Use the enhanced quantizer feature when quantizing the model. Regular quantization determines the range using the actual values of min and max of the data being quantized. Enhanced quantization uses an algorithm to determine optimal range. It can be useful for quantizing models that have long tails in the distribution of the data being quantized. [ --use_adjusted_weights_quantizer ] Note: Deprecated; use --param_quantizer. Use the adjusted tf quantizer for quantizing the weights only. This might be helpful for improving the accuracy of some models, such as denoise model as being tested. This option is only used when quantizing the weights with 8 bit. [ --optimizations= ] Note: Deprecated; use --algorithms. Enables new optimization algorithms. Usage is: --optimizations --optimizations Available optimization algorithms are: "cle" - Cross layer equalization includes a number of methods for equalizing weights and biases across layers in order to rectify imbalances that cause quantization errors. [ --algorithms= ] Enables new optimization algorithms. Usage is: --algorithms --algorithms Available optimization algorithms are: "cle" - Cross layer equalization includes a number of methods for equalizing weights and biases across layers in order to rectify imbalances that cause quantization errors. [ --override_params ] Use this option to override quantization parameters when quantization was provided from the original source framework (eg TF fake quantization). Note: Quantizer throws an error if overridden encodings contain unsupported bitwidths. [ --use_encoding_optimizations ] Note: Deprecated. Use this option to enable quantization encoding optimizations. This can reduce requantization in the graph and may improve accuracy for some models. [ --udo_package_path= ] Specifies the path to the registration library for UDO package(s). Usage is: --udo_package_path= You can (optionally) provide multiple packages as a comma-separated list. This option must be specified for networks with UDO. All UDO's in a network must have a host-executable CPU implementation. [ --use_symmetric_quantize_weights ] Note: Deprecated, use --param_quantizer. Use the symmetric quantizer feature when quantizing the weights of the model. It makes sure min and max have the same absolute values about zero. Symmetrically quantized data will also be stored as int#_t data such that the offset is always 0. [ --use_native_dtype ] Note: This option is deprecated, use --use_native_input_files option in future. Use this option to indicate how to read input files, 1. float (default): reads inputs as floats and quantizes if necessary based on quantization parameters in the model. 2. native: reads inputs assuming the data type to be native to the model. For ex., uint8_t. [ --use_native_input_files ] Use this option to indicate how to read input files, 1. float (default): reads inputs as floats and quantizes if necessary based on quantization parameters in the model. 2. native: reads inputs assuming the data type to be native to the model. For ex., uint8_t. [ --use_native_output_files ] Use this option to indicate the data type of the output files, 1. float (default): generates the output file as float data. 2. native: generates the output file as datatype native to the source model. i.e. uint8_t. [ --float_fallback ] Enables fallback to floating point (FP) instead of fixed point. This option can be paired with --float_bitwidth to indicate the bitwidth for FP (by default 32). If this option is enabled, then input list must not be provided and --override_params must be provided. The external quantization file (encoding file) might be missing quantization parameters for some interim tensors. First it will try to fill the gaps by propagating across math-invariant functions. If the quantization parameters are still missing, it applies fallback to nodes to floating point [ --param_quantizer= ] Indicates the weight/bias quantizer to use. Optional and must be followed by one of the following options: "tf": Uses the real min/max of the data and specified bitwidth (default). "enhanced": Uses an algorithm useful for quantizing models with long tails present in the weight distribution. "symmetric": Ensures min and max have the same absolute values about zero. Data will be stored as int#_t data such that the offset is always 0. [ --act_quantizer= ] Indicates the activation quantizer to use. Optional and must be followed by one of the following options: "tf": Uses the real min/max of the data and specified bitwidth (default). "enhanced": Uses an algorithm useful for quantizing models with long tails present in the weight distribution. "symmetric": Ensures min and max have the same absolute values about zero. Data will be stored as int#_t data such that the offset is always 0. [ --bitwidth= ] Note: Deprecated. Selects the bitwidth to use when quantizing the weights/activations/biases; 8 (default) or 16. Cannot be mixed with --weights_bitwidth or --act_bitwidth or --bias_bitwidth. [ --weights_bitwidth= ] Selects the bitwidth to use when quantizing the weights; either 4, 8 (default) or 16. 8w/16a is only supported by HTA currently. Cannot be mixed with --bitwidth. [ --act_bitwidth= ] Selects the bitwidth to use when quantizing the activations; either 8 (default) or 16. 8w/16a is only supported by HTA currently. Cannot be mixed with --bitwidth. [ --float_bitwidth= ] Selects the bitwidth to use when using float for parameters (weights/biases) and activations for all ops or a specific op (via encodings) selected through encoding; either 32 (default) or 16. [ --bias_bitwidth= ] Selects the bitwidth to use when quantizing the biases; either 8 (default) or 32. Using 32-bit biases may sometimes provide a small improvement in accuracy. Cannot be mixed with --bitwidth. [ --float_bias_bitwidth= ] Specifies the bitwidth for float bias tensors; either 32 or 16. If not provided and bias is overridden to float in the quantizer, the overriding float tensor's bitwidth will be used. [ --axis_quant ] Note: Deprecated; use --use_per_channel_quantization. Selects per-axis-element quantization for the weights and biases of certain layer types. Only Convolution, Deconvolution, and FullyConnected are supported. [ --use_per_channel_quantization ] Selects per-axis-element quantization for the weights and biases of certain layer types. Only Convolution, Deconvolution, and FullyConnected are supported. [ --use_per_row_quantization ] Enables row wise quantization of Matmul and FullyConnected ops. [ --enable_per_row_quantized_bias ] Enables row wise quantization of bias for FullyConnected op, when weights are per-row quantized. [ --restrict_quantization_steps= ] Specifies the number of steps to use for computing quantization encodings such that scale = (max - min) / number of quantization steps. The option should be passed as a comma separated pair of hexadecimal string minimum and maximum values, i.e., --restrict_quantization_steps "MIN,MAX". Note that this is a hexadecimal string literal and not a signed integer, to supply a negative value an explicit minus sign is required, e.g., --restrict_quantization_steps "-0x80,0x7F" indicates an example 8-bit range. --restrict_quantization_steps "-0x8000,0x7F7F" indicates an example 16-bit range. This option only applies to symmetric parameter quantization. Description: Generate 8 or 16 bit TensorFlow style fixed point weight and activations encodings for a floating point DLC model. Copy to clipboard Additional details: - For specifying input\_list, refer to input\_list argument in [snpe-net-run](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-net-run) for supported input formats (in order to calculate output activation encoding information for all layers, **do not** include the line which specifies desired outputs). - The tool requires the batch dimension of the DLC input file to be set to 1 during the original model conversion step. - An example of quantization using snpe-dlc-quantize can be found in the C/C++ Tutorial section: [Running the Inception v3 Model](https://docs.qualcomm.com/doc/80-63442-2/topic/tutorial_inceptionv3.html). For details on quantization see [Quantized vs Non-Quantized Models](https://docs.qualcomm.com/doc/80-63442-2/topic/quantized_models.html). - Using snpe-dlc-quantize is mandatory for running on HTA. - Using snpe-dlc-quantize is mandatory for running on DSP runtime on Snapdragon 865. It is recommended that offline cache generation be used. It is specified by using **–enable\_htp** option for snpe-dlc-quantize. - When using offline cache generation for HTP, the same input(s) tensors or layers and output(s) tensors or layers should be specified when using snpe-dlc-quantize and to run inference on the model using Qualcomm® Neural Processing SDK APIs or snpe-net-run. Not doing so will cause the cache to be invalidated, and graph initialization will take longer. - Outputs can be specified for snpe-dlc-quantize by modifying the input\_list in the following ways: > > > #[] > %[] > :=[:=] > … > Copy to clipboard > > > **Note:** Output tensors and layers can be specified > individually, but when specifying both, the order shown must > be used to specify each. - When running a model with an offline generated cache using snpe-net-run: > > > - Any output layers specified when snpe-dlc-quantize was > called, need to be specified using the input list as > shown in the input\_list argument to > [snpe-net-run](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-net-run). > - Any output tensors specified when snpe-dlc-quantized was > called, need to be specified using the > *–set\_output\_tensors* argument to snpe-net-run. Refer to > [snpe-net-run](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html#snpe-net-run) for > documentation. - When using the Qualcomm® Neural Processing SDK API: > > > - Any output layers specified when snpe-dlc-quantize was > called, need to be specified using the > Snpe\_SNPEBuilder\_SetOutputLayers() > function. > - Any output tensors specified when snpe-dlc-quantize was > called, need to be specified using the > Snpe\_SNPEBuilder\_SetOutputTensors() > function. ### snpe-udo-package-generator DESCRIPTION: ------------ This tool generates a UDO (User Defined Operation) package using a user provided config file. USAGE: ------------ snpe-udo-package-generator [-h] --config_path CONFIG_PATH [--debug] [--output_path OUTPUT_PATH] [-f] OPTIONAL ARGUMENTS: ------------------- -h, --help show this help message and exit --debug Returns debugging information from generating the package --output_path OUTPUT_PATH, -o OUTPUT_PATH Path where the package should be saved -f, --force-generation This option will delete the existing package Note appropriate file permissions must be set to use this option. REQUIRED_ARGUMENTS: ------------------- --config_path CONFIG_PATH, -p CONFIG_PATH The path to a config file that defines a UDO. Copy to clipboard ### qairt-quantizer The **qairt-quantizer** tool converts non-quantized DLC models into quantized DLC models. Basic command line usage looks like: usage: qairt-quantizer --input_dlc INPUT_DLC [--output_dlc OUTPUT_DLC] [--input_list INPUT_LIST] [--enable_float_fallback] [--apply_algorithms ALGORITHMS [ALGORITHMS ...]] [--bias_bitwidth BIAS_BITWIDTH] [--act_bitwidth ACT_BITWIDTH] [--weights_bitwidth WEIGHTS_BITWIDTH] [--float_bitwidth FLOAT_BITWIDTH] [--float_bias_bitwidth FLOAT_BIAS_BITWIDTH] [--ignore_quantization_overrides] [--use_per_channel_quantization] [--use_per_row_quantization] [--enable_per_row_quantized_bias] [--preserve_io_datatype [PRESERVE_IO_DATATYPE ...]] [--use_native_input_files] [--use_native_output_files] [--restrict_quantization_steps ENCODING_MIN, ENCODING_MAX] [--keep_weights_quantized] [--adjust_bias_encoding] [--act_quantizer_calibration ACT_QUANTIZER_CALIBRATION] [--param_quantizer_calibration PARAM_QUANTIZER_CALIBRATION] [--act_quantizer_schema ACT_QUANTIZER_SCHEMA] [--param_quantizer_schema PARAM_QUANTIZER_SCHEMA] [--percentile_calibration_value PERCENTILE_CALIBRATION_VALUE] [--use_aimet_quantizer] [--op_package_lib OP_PACKAGE_LIB] [--dump_encoding_json] [--config CONFIG_FILE] [--export_stripped_dlc] [-h] [--target_backend BACKEND] [--target_soc_model SOC_MODEL] [--debug [DEBUG]] required arguments: --input_dlc INPUT_DLC, -i INPUT_DLC Path to the dlc container containing the model for which fixed-point encoding metadata should be generated. This argument is required optional arguments: --output_dlc OUTPUT_DLC, -o OUTPUT_DLC Path at which the metadata-included quantized model container should be written.If this argument is omitted, the quantized model will be written at _quantized.dlc --input_list INPUT_LIST, -l INPUT_LIST Path to a file specifying the input data. This file should be a plain text file, containing one or more absolute file paths per line. Each path is expected to point to a binary file containing one input in the "raw" format, ready to be consumed by the quantizer without any further preprocessing. Multiple files per line separated by spaces indicate multiple inputs to the network. See documentation for more details. Must be specified for quantization. All subsequent quantization options are ignored when this is not provided. --enable_float_fallback, -f Use this option to enable fallback to floating point (FP) instead of fixed point. This option can be paired with --float_bitwidth to indicate the bitwidth for FP (by default 32). If this option is enabled, then input list must not be provided and --ignore_quantization_overrides must not be provided. The external quantization encodings (encoding file/FakeQuant encodings) might be missing quantization parameters for some interim tensors. First it will try to fill the gaps by propagating across math-invariant functions. If the quantization params are still missing, then it will apply fallback to nodes to floating point. --apply_algorithms ALGORITHMS [ALGORITHMS ...] Use this option to enable new optimization algorithms. Usage is: --apply_algorithms ... The available optimization algorithms are: "cle" - Cross layer equalization includes a number of methods for equalizing weights and biases across layers in order to rectify imbalances that cause quantization errors. --bias_bitwidth BIAS_BITWIDTH Use the --bias_bitwidth option to select the bitwidth to use when quantizing the biases, either 8 (default) or 32. --act_bitwidth ACT_BITWIDTH Use the --act_bitwidth option to select the bitwidth to use when quantizing the activations, either 8 (default) or 16. --weights_bitwidth WEIGHTS_BITWIDTH Use the --weights_bitwidth option to select the bitwidth to use when quantizing the weights, either 4, 8 (default) or 16. --float_bitwidth FLOAT_BITWIDTH Use the --float_bitwidth option to select the bitwidth to use for float tensors,either 32 (default) or 16. --float_bias_bitwidth FLOAT_BIAS_BITWIDTH Use the --float_bias_bitwidth option to select the bitwidth to use when biases are in float, either 32 or 16 (default '0' if not provided). --ignore_quantization_overrides Use only quantizer generated encodings, ignoring any user or model provided encodings. Note: Cannot use --ignore_quantization_overrides with --quantization_overrides (argument of Qairt Converter) --use_per_channel_quantization Use this option to enable per-channel quantization for convolution-based op weights. Note: This will only be used if built-in model Quantization-Aware Trained (QAT) encodings are not present for a given weight. --use_per_row_quantization Use this option to enable rowwise quantization of Matmul and FullyConnected ops. --enable_per_row_quantized_bias Enables row wise quantization of bias for FullyConnected op, when weights are per-row quantized. --preserve_io_datatype [PRESERVE_IO_DATATYPE ...] Use this option to preserve IO datatype. The different ways of using this option are as follows: --preserve_io_datatype e.g. --preserve_io_datatype input1 input2 output1 The user may choose to preserve the datatype for all the inputs and outputs of the graph. --preserve_io_datatype --use_native_input_files Boolean flag to indicate how to read input files. If not provided, reads inputs as floats and quantizes if necessary based on quantization parameters in the model. (default) If provided, reads inputs assuming the data type to be native to the model. For ex., uint8_t. --use_native_output_files Boolean flag to indicate the data type of the output files If not provided, outputs the file as floats. (default) If provided, outputs the file that is native to the model. For ex., uint8_t. --restrict_quantization_steps ENCODING_MIN, ENCODING_MAX Specifies the number of steps to use for computing quantization encodings such that scale = (max - min) / number of quantization steps. The option should be passed as a space separated pair of hexadecimal string minimum and maximum valuesi.e. --restrict_quantization_steps "MIN MAX". Please note that this is a hexadecimal string literal and not a signed integer, to supply a negative value an explicit minus sign is required. E.g.--restrict_quantization_steps "-0x80 0x7F" indicates an example 8 bit range, --restrict_quantization_steps "-0x8000 0x7F7F" indicates an example 16 bit range. This argument is required for 16-bit Matmul operations. --keep_weights_quantized Use this option to keep the weights quantized even when the output of the op is in floating point. Bias will be converted to floating point as per the output of the op. Required to enable wFxp_actFP configurations according to the provided bitwidth for weights and activations Note: These modes are not supported by all runtimes. Please check corresponding Backend OpDef supplement if these are supported --adjust_bias_encoding Use --adjust_bias_encoding option to modify bias encoding and weight encoding to ensure that the bias value is in the range of the bias encoding. This option is only applicable for per-channel quantized weights. NOTE: This may result in clipping of the weight values --act_quantizer_calibration ACT_QUANTIZER_CALIBRATION Specify which quantization calibration method to use for activations supported values: min-max (default), sqnr, entropy, mse, percentile This option can be paired with --act_quantizer_schema to override the quantization schema to use for activations otherwise default schema(asymmetric) will be used --param_quantizer_calibration PARAM_QUANTIZER_CALIBRATION Specify which quantization calibration method to use for parameters supported values: min-max (default), sqnr, entropy, mse, percentile This option can be paired with --param_quantizer_schema to override the quantization schema to use for parameters otherwise default schema(asymmetric) will be used --act_quantizer_schema ACT_QUANTIZER_SCHEMA Specify which quantization schema to use for activations supported values: asymmetric (default), symmetric, unsignedsymmetric --param_quantizer_schema PARAM_QUANTIZER_SCHEMA Specify which quantization schema to use for parameters supported values: asymmetric (default), symmetric, unsignedsymmetric --percentile_calibration_value PERCENTILE_CALIBRATION_VALUE Specify the percentile value to be used with Percentile calibration method The specified float value must lie within 90 and 100, default: 99.99 --use_aimet_quantizer Use AIMET for Quantization instead of QNN IR quantizer --op_package_lib OP_PACKAGE_LIB, -opl OP_PACKAGE_LIB Use this argument to pass an op package library for quantization. Must be in the form and be separated by a comma for multiple package libs --dump_encoding_json Use this argument to dump encoding of all the tensors in a json file --config CONFIG_FILE, -c CONFIG_FILE Use this argument to pass the path of the config YAML file with quantizer options --export_stripped_dlc Use this argument to export a DLC which strips out data not needed for graph composition -h, --help show this help message and exit --debug [DEBUG] Run the quantizer in debug mode. Backend Options: --target_backend BACKEND Use this option to specify the backend on which the model needs to run. Providing this option will generate a graph optimized for the given backend and this graph may not run on other backends. The default backend is HTP. Supported backends are CPU,GPU,DSP,HTP,HTA,LPAI. --target_soc_model SOC_MODEL Use this option to specify the SOC on which the model needs to run. This can be found from SOC info of the device and it starts with strings such as SDM, SM, QCS, IPQ, SA, QC, SC, SXR, SSG, STP, QRB, or AIC. NOTE: --target_backend option must be provided to use --target_soc_model option. Copy to clipboard ## Execution ### snpe-net-run snpe-net-run loads a DLC file, loads the data for the input tensor(s), and executes the network on the specified runtime. DESCRIPTION: ------------ Tool that loads and executes a neural network using the SDK API. REQUIRED ARGUMENTS: ------------------- --container Path to the DL container containing the network. --input_list Path to a file listing the inputs for the network. Optionally the file can have "#" starting line to specify the layer names or "%" to specify the output tensor names for which output tensor files are to be produced. For more details about the input_list file format, please refer to SDK html documentation (docs/general/tools.html#snpe-net-run input_list argument). OPTIONAL ARGUMENTS: ------------------- --use_gpu Use the GPU runtime for SNPE. Default float32 math and float16 storage. --use_dsp Use the DSP fixed point runtime for SNPE. Data & Math: 8bit fixed point Tensorflow style format. --use_aip Use the AIP fixed point runtime for SNPE. Data & Math: 8bit fixed point Tensorflow style format. --debug Specifies that output from all layers of the network will be saved. --output_dir= The directory to save output to. Defaults to ./output --storage_dir= The directory to store metadata files --encoding_type= Specifies the encoding type of input file. Valid settings are "nv21". Cannot be combined with --userbuffer*. --use_native_input_files Specifies to consume the input file(s) in their native data type(s). Must be used with --userbuffer_xxx. --use_native_output_files Specifies to write the output file(s) in their native data type(s). Must be used with --userbuffer_xxx. --userbuffer_auto Specifies to use userbuffer for input and output, with auto detection of types enabled. Must be used with user specified buffer. Cannot be combined with --encoding_type. --userbuffer_float Specifies to use userbuffer for inference, and the input type is float. Cannot be combined with --encoding_type. --userbuffer_floatN= Specifies to use userbuffer for inference, and the input type is float 16 or float 32. Cannot be combined with --encoding_type. --userbuffer_tf8 Specifies to use userbuffer for inference, and the input type is tf8exact0. Cannot be combined with --encoding_type. --userbuffer_tfN= Overrides the userbuffer output used for inference, and the output type is tf8exact0 or tf16exact0. Must be used with user specified buffer. --userbuffer_float_output Overrides the userbuffer output used for inference, and the output type is float. Must be used with user specified buffer. --userbuffer_floatN_output= Overrides the userbuffer output used for inference, and the output type is float 16 or float 32. Must be used with user specified buffer. --userbuffer_tfN_output= Overrides the userbuffer output used for inference, and the output type is tf8exact0 or tf16exact0. Must be used with user specified buffer. --userbuffer_tf8_output Overrides the userbuffer output used for inference, and the output type is tf8exact0. --userbuffer_uintN_output= Overrides the userbuffer output used for inference, and the output type is Uint N. Must be used with user specified buffer. --userbuffer_memorymapped Specifies to use memory-mapped (zero-copy) user buffer. Must be used with --userbuffer_float or --userbuffer_tf8 or userbuffer_tfN or userbuffer_auto etc. Cannot be combined with --encoding_type or --userbuffer_memorymapped_shared. Note: Passing this option will turn all input and output userbuffers into memory mapped buffer. --userbuffer_memorymapped_shared Specifies to use memory-mapped (zero-copy) user buffer with shared memory chunk. Must be used with --userbuffer_float or --userbuffer_tf8 or userbuffer_tfN or userbuffer_auto etc. Cannot be combined with --encoding_type or --userbuffer_memorymapped. Note: Passing this option will turn all input and output userbuffers into memory mapped buffer. --static_min_max Specifies to use quantization parameters from the model instead of input specific quantization. Used in conjunction with --userbuffer_tf8. --resizable_dim= Specifies the maximum number that resizable dimensions can grow into. Used as a hint to create UserBuffers for models with dynamic sized outputs. Should be a positive integer and is not applicable when using ITensor. --userbuffer_glbuffer [EXPERIMENTAL] Specifies to use userbuffer for inference, and the input source is OpenGL buffer. Cannot be combined with --encoding_type. GL buffer mode is only supported on Android OS. --data_type_map= Sets data type of IO buffers during prepare. Arguments should be provided in the following format: --data_type_map buffer_name1=buffer_name1_data_type --data_type_map buffer_name2=buffer_name2_data_type Data Type can have the following values: float16, float32, fixedPoint8, fixedPoint16, int8, int16, int32, int64, uint8, uint16, uint32, uint64, bool8 Note: Must use this option with --tensor_mode. --tensor_mode= Sets type of tensor to use. Arguments should be provided in the following format: --tensor_mode itensor Data Type can have the following values: userBuffer, itensor --perf_profile= Specifies perf profile to set. Valid settings are "low_balanced" , "balanced" , "default", "high_performance" ,"sustained_high_performance", "burst", "low_power_saver", "power_saver", "high_power_saver", "extreme_power_saver", and "system_settings". --perf_config_yaml Specifies the path to the yaml file containing the perf profile settings. --profiling_level= Specifies the profiling level. Valid settings are "off", "basic", "moderate", "detailed", and "linting". Default is detailed. --enable_cpu_fallback Enables cpu fallback functionality. Defaults to disable mode. --input_name= Specifies the name of input for which dimensions are specified e.g. --input_name= If the DLC has multiple graphs, graph names are required. Quotes are required if graph names are specified. Graph names can be found in snpe-dlc-info. e.g. --input_name=" " --input_dimensions= Specifies new dimensions for input whose name is specified in input_name. e.g. --input_dimension=1,224,224,3 For multiple inputs, specify --input_name= and --input_dimensions= multiple times. If the DLC has multiple graphs, graph names are required. Quotes are required if graph names are specified. Graph names can be found in snpe-dlc-info. e.g. --input_dimensions=" 1,224,224,3" --gpu_mode= Specifies gpu operation mode. Valid settings are "default", "float16". default = float32 math and float16 storage (equiv. use_gpu arg). float16 = float16 math and float16 storage. --enable_init_cache Enable init caching mode to accelerate the network building process. Defaults to disable. --platform_options= Specifies value to pass as platform options. --priority_hint= Specifies hint for priority level. Valid settings are "low", "normal", "normal_high", "high". Defaults to normal. Note: "normal_high" is only available on DSP. --inferences_per_duration= Specifies the number of inferences in specific duration (in seconds). e.g. "10,20". --runtime_order= Specifies the order of precedence for runtimes. Valid values are: cpu (Snapdragon CPU), gpu_float16 (Adreno GPU), gpu (Adreno GPU), aip (Snapdragon HTA+HVX), and dsp (Hexagon DSP). This option cannot be passed when any variant of --use_ is used. --set_output_tensors= Optionally, Specifies a comma separated list of tensors to be output after execution. If using Multi Graph DLC, use --set_output_tensors for each graph. e.g --set_output_tensors="graphA tensorA1,tensorA2" --set_output_tensors="graphB tensorB1,tensorB2" graph name is specified in snpe-dlc-info and defaults to the name of the first graph in the DLC. --set_unconsumed_as_output Sets all unconsumed tensors as outputs. aip_fixed8_tf (Snapdragon HTA+HVX) = Data & Math: 8bit fixed point Tensorflow style format cpu (Snapdragon CPU) = Same as cpu_float32 gpu (Adreno GPU) = Same as gpu_float32_16_hybrid dsp (Hexagon DSP) = Same as dsp_fixed8_tf aip (Snapdragon HTA+HVX) = Same as aip_fixed8_tf --udo_package_path= Path to the registration library for UDO package(s). Optionally, user can provide multiple packages as a comma-separated list. --duration= Specified the duration of the run in seconds. Loops over the input_list until this amount of time has transpired. --keep_num_outputs= Specifies the number of output sets to be generated. Loops over the input_list until this amount of outputs have been saved. --enable_cpu_fxp Enable the fixed point execution on CPU runtime --dbglogs --timeout= Execution terminated when exceeding time limit (in microseconds). Only valid for HTP (dsp v68+) runtime. --userlogs= Specifies the user level logging as level,. Valid values are: "warn", "verbose", "info", "error", "fatal" --model_name= Specifies model name for logging. --cache_compatibility_mode= Specifies the cache compatibility check mode; valid values are: "permissive" (default), "strict", and "always_generate_new". Only valid for HTP (dsp v68+) runtime. --validate_cache Perform an additional validation step just before building SNPE to check the validity of the selected cache record in the DLC. Upon success, app will proceed as usual. On validation failure, the app will report the validation error before exiting. --graph_init= Optionally, Specifies a comma separated list of specified graphs in the current DLC that is set to be inited. e.g --graph_init graph1, graph2, graph3 --graph_execute= Optionally, Specifies a comma separated list of specified graphs in the current DLC that is set to be executed. e.g --graph_execute graph1, graph2, graph3 --memory_limit_hint= Specifies the memory limit in Mb that DSP can use for initializing a cache. --init_from_buffer Specifies to use the Snpe_DlContainer_OpenBuffer API to create DLContainer handle. --enable_htp_accelerated_init Enable accelerated initialization only for HTP runtime and offline prepared dlcs. --help Show this help message. --version Show SDK Version Number. Copy to clipboard This binary outputs raw output tensors into the output folder by default. Examples of using snpe-net-run can be found in [Running the Inception v3 Model](https://docs.qualcomm.com/doc/80-63442-2/topic/tutorial_inceptionv3.html) tutorial. Additional details: - *Running batched inputs:* > > > - snpe-net-run is able to automatically batch the input > data. The batch size is indicated in the model container > (DLC file) but can also be set using the > “input\_dimensions” argument passed to snpe-net-run. Users > do not need to batch their input data. If the input data > is not batch, the input size needs to be a multiple of > the size of the input data files. snpe-net-run would > group the provided inputs into batches and pad the > incomplete batches (if present) with zeros. > > > In the example below, the model is set to accept batches > of three inputs. So, the inputs are automatically grouped > together to form batches by snpe-net-run and padding is > done to the final batch. Note that there are five output > files generated by snpe-net-run: > > > … > Processing DNN input(s): > cropped/notice_sign.raw > cropped/trash_bin.raw > cropped/plastic_cup.raw > Processing DNN input(s): > cropped/handicap_sign.raw > cropped/chairs.raw > Applying padding > Copy to clipboard - *input\_list argument:* > > > - snpe-net-run can take multiple input files as input data > per iteration, and specify multiple output names, in an > input list file formated as below: > > > #[] > :=[:=] > … > Copy to clipboard > > > The first line starting with a “#” specifies the output > layers’ names. If there is more than one output, a > whitespace should be used as a delimiter. Following the > first line, you can use multiple lines to supply input > files, one line per iteration, and each line only supply > one layer. If there is more than one input per line, a > whitespace should be used as a delimiter. > > > Here is an example, where the layer names are “Input\_1” > and “Input\_2”, and inputs are located in the path > “Placeholder\_1/real\_input\_inputs\_1/”. Its input list file > should look like this: > > > #Output_1 Output_2 > Input_1:=Placeholder_1/real_input_inputs_1/0-0#e6fb51.rawtensor Input_2:=Placeholder_1/real_input_inputs_1/0-1#8a171b.rawtensor > Input_1:=Placeholder_1/real_input_inputs_1/1-0#67c965.rawtensor Input_2:=Placeholder_1/real_input_inputs_1/1-1#54f1ff.rawtensor > Input_1:=Placeholder_1/real_input_inputs_1/2-0#b42dc6.rawtensor Input_2:=Placeholder_1/real_input_inputs_1/2-1#346a0e.rawtensor > Copy to clipboard > > > Similar to above a first line starting with “%” specifies the output tensor names > > > **Note:** If the batch dimension of the model is greater > than 1, the number of batch elements in the input file > has to either match the batch dimension specified in the > DLC or it has to be one. In the latter case, snpe-net-run > will combine multiple lines into a single input tensor. - *Running AIP Runtime:* > > > - AIP Runtime requires a DLC which was quantized, and HTA > sections were generated offline. > - AIP Runtime does not support debug\_mode > - AIP Runtime requires a DLC with all the layers > partitioned to HTA to support batched inputs - *Set cache compatibility mode:* > > > - A DLC can include more than one cache record and users can set > the compatibility mode to check whether the best cache record is > optimal for the device. The available modes indicate binary cache > compatibility as follows. > > > > > > > > > 1. permissive – Compatible if it could run on the device. > > 2. strict – Compatible if it could run on the device and > > fully utilize hardware capability. > > 3. always\_generate\_new – Always incompatible; SNPE will > > generate a new cache. ### snpe-parallel-run snpe-parallel-run loads a DLC file, loads the data for the input tensor(s), and executes the network on the specified runtime. This app is similar to snpe-net-run, but is able to run multiple threads of inference on the same network for benchmarking purposes. DESCRIPTION: ------------ Tool that loads and executes one or more neural networks on different threads with optional asynchronous input/output processing using SDK APIs. REQUIRED ARGUMENTS: ------------------- --container Path to the DL container containing the network. --input_list Path to a file listing the inputs for the network. --perf_profile Specifies perf profile to set. Valid settings are "balanced" , "default" , "high_performance" , "sustained_high_performance" , "burst" , "power_saver", "low_power_saver", "high_power_saver", "extreme_power_saver", "low_balanced", and "system_settings". NOTE: "balanced" and "default" are the same. "default" is being deprecated in the future. --cpu_fallback Enables cpu fallback functionality. Valid settings are "false", "true". --runtime_order Specifies the order of precedence for runtime e.g cpu,gpu etc. Valid values are: cpu, gpu, gpu_float16, dsp, aip. This option cannot be passed when any variant of --use_ is used. --use_cpu Use the CPU runtime for SNPE (Snapdragon CPU). Data & Math: float 32bit. Only one --use_ option is needed. --use_gpu Use the GPU float32 runtime (Adreno GPU). Data: float 16bit Math: float 32bit. --use_gpu_fp16 Use the GPU float16 runtime (Adreno GPU). Data: float 16bit Math: float 16bit. --use_dsp Use the DSP fixed point runtime (Hexagon DSP). Data & Math: 8bit fixed point Tensorflow style format. --use_aip Use the AIP fixed point runtime (Snapdragon HTA+HVX). Data & Math: 8bit fixed point Tensorflow style format. --perf_config_yaml Specifies the path to the yaml file containing the paths to the init, execute and dinit yaml files. NOTE: --perf_profile and --perf_config_yaml are mutually exclusive. Only one of the options can be specified at a time. OPTIONAL ARGUMENTS: ------------------- --userbuffer_float Specifies to use userbuffer for inference, and the input type is float. --userbuffer_tf8 Specifies to use userbuffer for inference, and the input type is tf8exact0. --userbuffer_auto Specifies to use userbuffer with automatic input and output type detection for inference. --use_native_input_files Specifies to consume the input file(s) in their native data type(s). Must be used with --userbuffer_xxx. --use_native_output_files Specifies to write the output file(s) in their native data type(s). Must be used with --userbuffer_xxx. --input_name Specifies the name of input for which dimensions are specified. --input_dimensions Specifies new dimensions for input whose name is specified in input_name. e.g. "1,224,224,3". --output_dir The directory to save result files --static_min_max Specifies to use quantization parameters from the model instead of input specific quantization. Used in conjunction with --userbuffer_tf8. --userbuffer_float_output Overrides the userbuffer output used for inference, and the output type is float. Must be used with user specified buffer. --userbuffer_tf8_output Overrides the userbuffer output used for inference, and the output type is tf8exact0. Must be used with user specified buffer. --userbuffer_memorymapped Specifies to use memory-mapped (zero-copy) user buffer. Must be used with --userbuffer_float or --userbuffer_tf8 or userbuffer_tfN or userbuffer_auto etc. Cannot be combined with --encoding_type. Note: Passing this option will turn all input and output userbuffers into memory mapped buffer. --userbuffer_memorymapped_shared Specifies to use memory-mapped (zero-copy) user buffer with shared memory chunk. Must be used with --userbuffer_float or --userbuffer_tf8 or userbuffer_tfN or userbuffer_auto etc. Cannot be combined with --encoding_type or --userbuffer_memorymapped. Note: Passing this option will turn all input and output userbuffers into memory mapped buffer --enable_init_cache Enable init caching mode to accelerate the network building process. Defaults to disable. --profiling_level Specifies the profiling level. Valid settings are "off", "basic", "moderate", "detailed", and "linting". Default is off. --platform_options Specifies value to pass as platform options. Valid settings: "HtaDLBC:ON/OFF", "unsignedPD:ON/OFF". --platform_options_local Specifies the value to pass for the current SNPE instance. Valid settings: "HtpDLBC:ON/OFF", "HtaDLBC:ON/OFF;HtpDLBC:ON/OFF". --runtime_mode Specifies the transmission mode for input/output processing. Valid settings are "sync", "output_async", "inputoutput_async". --set_output_tensors Specifies a comma separated list of tensors to be output after execution. --userlogs Specifies the user level logging as level,. --enable_cpu_fxp Enable the fixed point execution on CPU runtime. --init_from_buffer Specifies to use the Snpe_DlContainer_OpenBuffer API to create DLContainer handle. --enable_htp_accelerated_init Enable accelerated initialization only for HTP runtime and offline prepared dlcs. --version Show SDK Version Number. --help Show this help message. Copy to clipboard Additional details: - *Required runtime argument:* > > > - For the required arguments pertaining to runtime > specification, either –runtime\_order OR –use\_cpu OR –use\_gpu etc. > needs to be specified. The following example demonstrates > an equivalent command using either of these options. > > > > snpe-parallel-run --container container.dlc --input_list input_list.txt > --perf_profile burst --cpu_fallback true --use_dsp --use_gpu --userbuffer_auto > Copy to clipboard > > > is equivalent to > > > snpe-parallel-run --container container.dlc --input_list input_list.txt > --perf_profile burst --cpu_fallback true --runtime_order dsp,gpu --userbuffer_auto > Copy to clipboard - *Spawning multiple threads:* > > > - snpe-parallel-run is able to create multiple threads to > execute identical inference passes. In the example below, the given command has the required arguments for container and input list given. After these 2 options, the remaining options form a repeating sequence that corresponds to each thread. In this example, we have varied the runtimes specified for each thread (one for dsp, another for gpu, and the last one for dsp). snpe-parallel-run --container container.dlc --input_list input_list.txt --perf_profile burst --cpu_fallback true --use_dsp --userbuffer_auto --perf_profile burst --cpu_fallback true --use_gpu --userbuffer_auto --perf_profile burst --cpu_fallback true --use_dsp --userbuffer_auto Copy to clipboard When this command is executed, the following section of output is observed: ... Processing DNN input(s): input.raw PSNPE start executing... runtimes: dsp_fixed8_tf gpu_float32_16_hybrid dsp_fixed8_tf - Mode :0- Number of images processed: x Build time: x seconds. ... Copy to clipboard Note that the number of runtimes listed corresponds to the number of threads specified, as well as the order in which those threads were specified. ### snpe-throughput-net-run snpe-throughput-net-run concurrently runs multiple instances of SNPE for a certain duration of time and measures inference throughput. Each instance of SNPE can have its own model, designated runtime and performance profile. Please note that the –duration parameter is common for all instances of SNPE created. DESCRIPTION: ------------ Tool to load and execute concurrent SNPE objects using the SDK API. REQUIRED ARGUMENTS: ------------------- --container Path to the DL container containing the network. --duration Duration of time (in seconds) to run network execution. --use_cpu Use the CPU runtime for SNPE (Snapdragon CPU). Data & Math: float 32bit. Only one --use_ option is needed. --use_gpu Use the GPU float32 runtime (Adreno GPU). Data: float 16bit Math: float 32bit. --use_gpu_fp16 Use the GPU float16 runtime (Adreno GPU). Data: float 16bit Math: float 16bit. --use_dsp Use the DSP fixed point runtime (Hexagon DSP). Data & Math: 8bit fixed point Tensorflow style format. --use_aip Use the AIP fixed point runtime (Snapdragon HTA+HVX). Data & Math: 8bit fixed point Tensorflow style format. --perf_profile Specifies perf profile to set. Valid settings are "balanced" , "default" , "high_performance" , "sustained_high_performance" , "burst" , "power_saver", "low_power_saver", "high_power_saver", "extreme_power_saver", "low_balanced", and "system_settings". NOTE: "balanced" and "default" are the same. "default" is being deprecated in the future. --perf_config_yaml Specifies the path to the yaml config file containing the perf configs. --runtime_order Specifies the order of precedence for runtime e.g cpu,gpu etc. Valid values are: cpu, gpu, gpu_float16, dsp, aip. This option cannot be passed when any variant of --use_ is used. OPTIONAL ARGUMENTS: ------------------- --debug Specifies that output from all layers of the network will be saved. --userbuffer_auto Specifies to use userbuffer for input and output, with auto detection of types enabled. Must be used with user specified buffer. --userbuffer_float Specifies to use userbuffer for inference, and the input type is float. Must be used with user specified buffer. --userbuffer_floatN Specifies to use userbuffer for inference, and the input type is float16 or float32. Must be used with user specified buffer. --userbuffer_tf8 Specifies to use userbuffer for inference, and the input type is tf8exact0. Must be used with user specified buffer. --userbuffer_tfN Specifies to use userbuffer for inference, and the input type is tf8exact0 or tf16exact0. Must be used with user specified buffer. --userbuffer_memorymapped Specifies to use memory-mapped (zero-copy) user buffer. Must be used with --userbuffer_float or --userbuffer_tf8 or userbuffer_tfN or userbuffer_auto etc. Cannot be combined with --encoding_type or --userbuffer_memorymapped_shared. Note: Passing this option will turn all input and output userbuffers into memory mapped buffer. Note: Passing this option will turn all input and output userbuffers into memory mapped buffer. --userbuffer_memorymapped_shared Specifies to use memory mapped user buffer in a shared memory chunk. Must be used with --userbuffer_float or --userbuffer_tf8 or userbuffer_tfN or userbuffer_auto etc. Cannot be combined with --encoding_type or --userbuffer_memorymapped. Note: Passing this option will turn all input and output userbuffers into memory mapped buffer. --userbuffer_float_output Overrides the userbuffer output used for inference, and the output type is float. Must be used with user specified buffer. --userbuffer_floatN_output Overrides the userbuffer output used for inference, and the output type is float16 or float32. Must be used with user specified buffer. --userbuffer_tf8_output Overrides the userbuffer output used for inference, and the output type is tf8exact0. Must be used with user specified buffer. --userbuffer_tfN_output Overrides the userbuffer output used for inference, and the output type is tf8exact0 or tf16exact0. Must be used with user specified buffer. --storage_dir The directory to store metadata files --version Show SDK Version Number. --iterations Number of times to iterate through entire input list --verbose Print more debug information. --skip_execute Don't do execution (just graph build/teardown) --enable_cpu_fallback Enables cpu fallback functionality. Defaults to disable mode. --json Generated JSON report. --input_raw Path to raw inputs for the network, seperated by ",". --fixed_fps Fix fps so as to control system loading, total FPS will be limited to around Ex: 30,20,0(free run) --udo_package_path Path to UDO package with registration library for UDOs. Optionally, user can provide multiple packages as a comma-separated list. --enable_init_cache Enable init caching mode to accelerate the network building process. Defaults to disable. --platform_options Specifies value to pass as platform options for all SNPE instances. --platform_options_local Specifies the value to pass as per SNPE instance platform options for the current SNPE instance. if --platform_options is specified then it overwrites the global platform options for the current SNPE instance. --priority_hint Specifies hint for priority level. Valid settings are "low", "normal", "normal_high", "high". Defaults to normal. Note: "normal_high" is only available on DSP. --groupDuration Duration (in ms) of execution before next sleep.(Optional) --groupSleep Sleep interval (in ms) after execution of group.(Optional) --set_output_layers Optionally, user can provide a comma separated list of layers to be output after execution. If using multi-graph DLC, provide in double quotes. It should be defined for all instances or none at all. Use empty string for instances that doesn't need any layer outputs. e.g --set_output_layers "graphA layer1,layer2,layer3" --set_output_tensors Optionally, user can provide a comma separated list of tensors to be output after execution. If using multi-graph DLC, provide in double quotes. It should be defined for all instances or none at all. Use empty string for instances that doesn't need any layer outputs. e.g --set_output_tensors "graphA tensor1,tensor2,tensor3" --userlogs= Specifies the user level logging as level,. Valid values are: "warn", "verbose", "info", "error", "fatal" --enable_cpu_fxp Enable the fixed point execution on CPU runtime. --model_name To add the model name to the logs (Optional) --cache_compatibility_mode= Specifies the cache compatibility check mode; valid values are: "permissive" (default), "strict", and "always_generate_new". Only valid for HTP (dsp v68+) runtime. --validate_cache Perform an additional validation step just before building SNPE to check the validity of the selected cache record in the DLC. Upon success, app will proceed as usual. On validation failure, the app will report the validation error before exiting. --graph_init Optionally, Specifies a comma separated list of specified graphs in the current DLC that is set to be inited. e.g --graph_init graph1, graph2, graph3 --graph_execute Optionally, Specifies a comma separated list of specified graphs in the current DLC that is set to be executed. e.g --graph_execute graph1, graph2, graph3 --enable_htp_accelerated_init Enable accelerated initialization only for HTP runtime and offline prepared dlcs. --help Show this help message. Copy to clipboard ## Analysis ### snpe-diagview snpe-diagview loads a DiagLog file generated by snpe-net-run whenever it operates on input tensor data. The DiagLog file contains timing information information for each layer as well as the entire forward propagate time. If the run uses an input list of input tensors, the timing info reported by snpe-diagview is an average over the entire input set. The snpe-net-run generates a file called “SNPEDiag\_0.log”, “SNPEDiag\_1.log” … , “SNPEDiag\_n.log”, where n corresponds to the nth iteration of the snpe-net-run execution. usage: snpe-diagview --input_log DIAG_LOG [-h] [--output CSV_FILE] Reads a diagnostic log and output the contents to stdout required arguments: --input_log DIAG_LOG Diagnostic log file (required) optional arguments: --output CSV_FILE Output CSV file with all diagnostic data (optional) --chrometrace CHROMETRACE_FILE Output chrometrace JSON filename (w/out extension) for logs made using profiling level linting (optional) --csv_format_version CSV_FORMAT_VERSION Indicates to the tool which CSV format version to output CSVs in. Valid options are: 1 (default), 2. This flag is only meaningful in conjunction with the --output flag Copy to clipboard The output generated has timings collected at different layers of the stack. Below is the description of the timing markers: (Please note: Certain backends like DSP or GPU sometimes splits an op into multiple ops in the backend. Sometimes the backends fuse multiple ops into one. Hence a detailed profiling log can display mismatching no of ops and layer mapping.) ![../images/snpe_diagview_timing_diagram.png](data:image/png;base64,UklGRs4pAABXRUJQVlA4TMEpAAAvKASPAPWK4rZto2j/uXM6ab8RwTAA0jaQ7gVo2x/sn2TJiuO83CL3OUbymJW5z8/0EDfAlh5kVT8a6C9RvgwVuGA3PIdq4Ve2tLb4yvvZwMQ4C3skLC07RracHVhSfTWIECRFxyJG73KHPWyeFgqToaivsCeTlirIZe54hKAeYF5uIGGg+tVf6VdoWKftkyRH0r/RcKFgw4GCCwUFBRs2FGw4UHDhvSso2FBQUFCwoKCg2JiF/34/94iM9PAIT3Voau9j7lNmNVm2Zb73ltmiO/1Xe6bPfbjN6aZjLq35pHbOcqvy1ZhsoAhgG0fy//FcsHBgYGDgQENBzS0oqNkTxm1td2OpTzu3Yz1zf9fQh8FisRgMfhgMBoMfFqf/tGDbChpLm3QG0sg7GL0iotyeT78LQH1ym+3fv+HS5V2mTJky5V2mTJkyZUqXLlOmvMuULu/SpcuUKV2mzN8w17mumR3NdUZ47x8Cu5LQLfAUrvYcdkDFFlsIZFjBERajhRWzjAZkRnsjOAYbZmECgvwQjNiKSNum/lVd0uGg0bZtc4GGhYPrvyRIkuS2qeUFGCC798ZiQdHf91RWJ8LsP7P/zP4z+8/sP7P/zP4z+8+DLNwMF1QwVGWuBy4dcKfgq5YWypBrhlEQXJFsj+HYGxxzkcHeFhOhjWHfEzzHi9ZBUOsocLDGYHOyeLD+yF5SE852GWhVAnXyAxQgPJQAT1+AtXGozwMIKCQ5iw9tNp4dsDYgBKLLxAnEDzBeIBRCXhb2CCmVVCMzjbPHCcraVJDVCw2ZI7AttQH+VLwcwAtpNUVo1yiCUBS5SLLY5sUceb1ENGyIjyPTDZPCFVmbSCjwFMjEOQqhyTZ/HClzJcwSBg/ucMT1W4KScpZRoLQL+mJPdsvaZoCtyayG56wO00jEWW2EWQpKiZKi0BOJAR70G9NQcjlbbAI71PlZmWVZQH8pyAKkFAqYxTMCCEe1KBYaaoA3pUFaUAIfRdzZSpJYUlBmojbOXuMIEcWxfKBXiuJVoRQSJM8jCnXJJnORszoRZv+ZhSCDARLosXXU7eKeUSj0XRzsScF4/DdxiwccFG7KFC6UgMw4ZOK+XUYQW1DpEMdTBB5v2PDhNFUFjqrURqwMDouiYOPXpVHNRsK+FPYyOEw7OI6oVQLIJAHbIdUtNnBB9owd3hH8guHSa6gdN0paX6SorDSNQZf7tNfgW23kjzVGIAHMddKBvuMzcOAoAYqoDYqKOUi0Ry4LoyCrsCSAtUWKIFUUKRx1TuJMgJVOOmSm2yY+s+8CYnMP3C9SYN9zKtHgkrS+xMrP+OkbmIhVDQuVuKOH+JYL3FaEmohblQFQqWckxsiZ0TbGo53I7EgqOylFMQGlFI4ktUksHKLF4aEVQusgchVqKpYVARE1RIQsx1csvrPNtYGwozKjGrEtrhwIsFaN4xQSmWrPBQm4HUc196jzUGypRLXgljbSA9oQqTOdwEWepZBZAsZzIA6WZHfr4sDmgahikUD1CgbU50socNhSOdQIq11QZWSyvSLH0wd/x1h34VKEGNWDv5AV+LLATwDLfQVvGArIFqS2IKihgvVTZQShwKGmW2lAX4bjKihCsQM6/FjPRW3SFQA2TBkpqRG5EuaA7N1fcBYHNcA1IQOazZgqlNXgsXEEeHyQ5dxJQtr6ohs/0oQjD/4yYKiSg3+YAge/lchlSQlJTkYtEVASDdEgLVzIZhkpC+YFX8TYD4UzorLDH7dr0GdyNyC22oXDAbVEQEmyBKpS5Ep9TMtCeUubakdqh5a0vsQaVgDv0pKDv4woFATFg7+g4qZZ5qX9GEGevAbxtSIpSmShGUJsWgioEFR29IdANIIDly6oRLtaxqI/BqcfWIoKcK+CEEdoigDbYO0JKUlaX+1gRx/8JZQEFCse/LGYzCCrNkiO2xMoIjMN3Lwcj/mlSkrSvZu5CwJXrLtHiAq0x1E4JsFVZtd+MUe073EQiRYfR3KfVsIpHfwJgKbIFRM8lVB+KxhmUCmSwJXDeMcz3Ppik8AU5lbIL/nl3iA5loqSlN8qorKjZUE1TQ1aA05MZChnQVVpjfTgT4G15pzQIHKmz9+iWLeFacD5GzD9ycMvJpNUdlzHfAlE258gORXIrgR0JCQU1R7hwBkEw7UBqiTWhYQKZGXOqOqeSSI7Th0QOUqJYBXDmsnhRB381d6UDv4EYNVx1AEwgLyROO+r4JxGRgrEiWyWykJf96gPWeBSovAdkImdoHDuA0RHC1XFgm2nBEJWxrsdiZASusBzpk5zyD0PHuBlBHb0wT/A1UGjdPAnKezY64MMVymwVFhBLBLgqzhG26MrCMK0T3yhkN2byC4RCmqQAF81Z7TfRTIB2j4tkKiT94dQ5hdDmD/q4I9XkASlgz8EV8gNy/bh01QcQk6dUNgpdJxGdKiX4iQHf/gqkoTSwZ8mcBkSnEuTMTpqkkcelv/G6fi/XPVDDtpH0rhLEzFRS/0QkcugUCdkNHSOwDlZyqAjfFqdCHPYXML/VhuV13r1dbrSAnvyoa3UbUXU05FkPcTMvDrAvgRygAwhJdp/kBVC1ijjW+sqwjHj0V+rBkjvMim4TePZrTjguCrc8+sb2FUgAJwM2Xw8cyUsFkbyjCNmZqID1o1qbmf6Y21UHnX7mEZaIK6wyGHpghLjU5jxLT5QWeaClrOzIRnUpEbgikoCeoiHN3KwtGz5ahxkQmh1hQukEDOi2X2ILFMDd+ORNiqPun00Ii0xqBSklIhWvyCF/zC6/lLoLMRECdCIrQRA3XZCFoTxuCfW5qN8M1rEEthR5ivbl0SKR9o7UzDdF6QybrJdP0NFsJMr4rhgHFVxUb3urilbtUCyv0AK94R9CrsZqsr9E5lbIBfUt/lHbTZ+OJnwb29Xw8GRCl6chER0PhTlEO5wQn2MxkIV7MRrkA2BaBQ4YhhEIHzVUABVLzTSQeWFSso9BOm9+XZtMSVwbyoE928GBfD94RCkgzBMAQuuRE0CMbAyJ0KVMkgpHpYZlR63u93aPgJocjh2MFHRzVhZiSgsR0AJYWzAcgIvdOWElsMNI1GdAKpWCFUtqMKaoYp7UROwnFUkMypJ8rFjKdgxHEzlmNDtnZEr1WESm5LYYenQIElUqSeAUlhJVMembsUm4tlhC4KAj6lsgYNERsEJ6GfcFQKP+l8kqrogp0A8L693bLc2icyDxmU8tu/5ZMLgDRzPtFF5JOx3kCgcQKI15OkIkcBVokAJiapuKFzVvrt+cL/eki0JLbO4hsgF5GqDJEvkIKA6SDNXCioLpaoTcPlBOsLDrtGqIHDp1KHfd0HF+hAp4LMUCqIgUlU3pIL19o4rPbb382ZxgEMrHJTCDhqe4jogJfh7DLIqJZHggV6kqq5IHv6+f1zBaFhdCBPKqNKbvv6jNiqPuv1BWWm65ta0C7qn3JzudYa/IOTPrw85NzbvND80afbIrcn1k796qu9MthwcvD38zn3HlNkTTew3d04AHB1pmxMQMWHuMiJ2MgWwv71XF21zAiL3E9Plq7cWLw8OTn7j8LGbdtntzDfyFrer3G5OD5pY8w2L19KGNy7bPxzQQb/VT5FIu9xjq+t1LvYaGLlxdq9jsK89cmMIfiAjL5gB1zdvXzDZ6A/3wMsmghLjLphA4v3/d89JTTDfyLQLiN1uQMyEI+OPQS/IiHNc1/yKjHxniSuvje8A+Qw05AO9UMxwudyPLdmdOt0OHTeUrtN+PyPl6WRLEzNt0shj0G9vty30ZLEJg+9tP70qwTX1F9qoPOr2tbHS+oIVD9aSwCjExen2kEJ4ldPLVUPUKXKxyHDtqcc6nbLdJjuF74vugkxH5CtysSjPLlI7n03AmmRZblLmxXKR6yV0dSakfjOjKGhxOGY0IVAoSEJlB6xPQPOQEaqEJGanSOUYOUNlXGDtkOiUMnFaBOrmOEGBUOh2ywWdP3klRKI0tQKyHKN2Dzhls2rgXlBki4CkyIz+rlGpKtwZIieg+c8aIjGxYaA0EvOsE1O0J6IKCCyzqMOgXg5VCq0RZ79AVoScQFkQkclxoNWRcyOgApkFqUpAUCUhFmwQkj1WVgW+EQ3mWI64ZQS0FEEW4kXN4EqrjFo36gjtGdQKqi4JLEdNXmS3oF0y8MAceGBFIlguEIYiChVQgeJijtBUGmWwJKFO5ks7O9z0cdgMCSGQiM+IqnoiYXGOiZXCgSzIbiU8O6wE7fMO+E4Bl0M9J0xQuKhmIQjAwAiZAsIYRWTHIlcI4foJxhGXijvBU38/0DTCq1faqDwS9ntDZB/AJwH2X0iIi8IzIwJFZMZcxPLLXHqC1Rq/QScQngv8ZQ64/6KSb1EV5QThB1JBiUglmAnlwsxEomO3Gzxhx2wZkizIHOTMBS8HGwoOgbMEVkeaJYOKRWFRgHVxKItjxm0oVEmifMelG1D5hZ2NoQcFEeaAieWcLIHPU1qhVBCl1tFiMk4ACQGLc6kjvGwFpPShDyXbRXLQKhBUdnA5LCpXBLY/MbQiVUFNgipUE1ZLtF3gqzZZ4MsVQQEGrUJT2BFIdk50sCbe6CFVDSqHzP1cadakIFyRNkG22xUAVC+gOA7mwA9HOp95eWI+bxn1fUoZ8MmL7orjsxHO2c1l45iSASUcXLpHaL+gzxwc3BycPB3XnAnHlJYcVINZcNJE7oD7lXHstiLOcXLSYGxuKw47JyAykh2D/tg3XhqfdxjDzm4HInud9ViF7jql/M2Zdjy9OT4bvZwdjwPeXOuP9Y9wiXXshj/aQ747EhxT0nzi7bPfuRdG8WI/fAocUzK5xYYd6LTiw3FcEI5MNIAc951RPdDrh09vTg4mGYBjzaN56DjtmJKxMeDTyN8VTdfpOL8Z7X4yvYwIffrD0X7XqtljTf54fOQz0AqFAycP6OMlc/0EPPEPLaoSZGIy1OVx9ebVqzRhcHRkEr5f57lHMEvsB21N3KI4OurA1Zsha+VOz4vBiQLQPETgsVmUiIjyM0dMXsBK8yq+E1TqmZ52Fs8OQIjwRHsl86oBQUYkJyBVwcGJjPAkiJwl0xM5BnAt4vymojmKVGYeEp8HPLOZI07Biu1MFIoMZ0FuRQpwtkJPP/o7oikSFXBQUFUB9b5kNI9ZYRACNZWhTQQHR2fJSRgZjEKWCER48KvYKIbLeWiCmClgonBOYDk1SkrAvHkhCyuvVEH2kZRIYuoCj1ocDktOBZ8kScGNosheKZGwrpzxDKueI4VFLCgkopVBn+a0TCADA5ZFV1mZa+BKc/RuF4Q0bKUlnCLRrLKRA70IaxEpsJK2OHx+okyKdhrPWUYHBUWG/3FYjEioEoizp9ZUonSLSXLwAquXnNJU+BNUAJansoNZQLGeyhK4YN0oC6xAbQsuP3Kg3Fa0Jre3LxGFA90fIrnjRGXoeAH/FjREjuhYjUgBvLr6Oq/fFgAdAV9tbU9FXbqTyfgNJMyn6/wN1BSRd5x62OeRlGB4ekPZpUr7gwNelzG2F9RqI4oal1FSB6QkcAPgbsGkx2dengzpBqR/Tn7vxfQG7fTBmOLs4ODmdA9MbVBOHww0+9u/1TUnTeRsWoNw+mB8cdJEPnk2pUE4fTC+2O00ExnXTh+MNXcLfu/FRMbZ8fjjWB/RidPpAwNIfPcOuJ+kiSxInD4Y6uHB2s6CYuPYyTQk7NNOHwz0BqRtYGxSG+hFOX0wsgZ6gdhUN3T87HhcDR0/Ppv0bkZ7OsgcHWmaJv+kRxpTA72SncNy/UExKRJpVzi293JJVjys1PBse7d8xwnuhaNkTf7UaxPYgCx5QasqKE1Q0CmFuSWZqRniHXtrTEG2JNxJ++M/6gzHXDrggr7sM01+mP0Kju1Rd02fmF17cZGLbU5UjugfuIyjOyLXyH5k86EPaaPyqNtHP9I8O2L8xq0UB8pVFKEgAwJWQMzbTLl9AmZgJcSpIBLic+Cvlscii5Drk20NjjkSEHMvOzxzMuWvlqTtkxlIQJ5dUOJUQb0vqLcWz99crOsL2Q2IE1YnA4B1iZS/WpROj+1PRg4t1Yg81grqeGHMMpRWRZ3QcjY2AzNIQJcwCC+12GEFgTHs4JkSebqEr6m0SWBPM3xD5AoXqsnKhsAheQQT5Ez6qyW5jLvPnnQ3h3PqDkecTnEcUVxTf6GNyqNuPz0iLbMyLIE0L33AkhYkP1nJWQo2J1n16gxnWaJLIlG6XaWgJoqTJxwQU0h/tWTjMgIXpSydAMqHlbKrI0KJE5B8y9M/3i4xhJpMCYtSd/kqO6WCuiFFJNfS3owWU7MYTSVMSKRzU2Ylc1EZ6NUR0LBFa3t7e8MR3b5JSxEs3WI9xFqH4eaH9obPvDxJWU5+766wLPTC5ca3sD57qGeEK3pWJdZHlxv1CfRa0aY8BwO73DjInICIPSnoDkUuNw4yux2M2BGu9Nj2wpO+IfBa0Zbx1pivzsWKs7+9TSmXGxdZkA1P7G//FrncOMiA2B6wJ3OeYpcbhxgRe2pRBoZSLjcOMAfvtSsx2uXGoeX4LFmbcD9ZGNfYP6Rnl5CeJjsX7Lg3ektP5th2qnYilMrB8k2OvNyA6cc64IK+VgII3Ml8Y5QNs738qpZr6ovUzi5xsLAsY0GFC5gUVeJKgQcnOCo0CfG50FfBQ8MBuREQLDiOE5zXkHKlAPebBmVIDwwcnqqA87T9x+PjEWlLjeeA7U8FOXyjAebI7JUJYEqjAqU5NNVZTBYsLDcRlBDEclH+Z+u3SMAzJeTnBTu4kLPdZCZObxx7wnesDqmVcbBiLPNilHnpBRY7wLKxTVpUoiYONq2gZYYz/seyWpZYQOVXzlYb0t8bx1WpoHjA/uTUxUxmaF1Y4vkQM2FWCmeVgV5xkdwCf/9sbk3uObaNz39oR8+HqNGwOnh0a3FXWLcRocR8iLrlQEQsG3g+RO1yciIid4nRkudomEfdPh6R1g8+huZD1C87GPlqsiKhH9xYTdfzIZ7gKcp+datvZFvG9dG0z8HN6WfsGDd8f/VwD/nugc4T10+efdWWfZzQJGc6zoJi9mSxPkX0WxCI2bgRoT98+t4D3fLyGx+zKkPHezcfolYRXad2jf9/9lDTMwR+SMm28Wyr7W5kO4cdD/MufOQf/0gDWNIZylJKlnyxVXTeKF4LOg6XMLavRWOo/JMnmufPtl9/YqRhJ5k8ee2HSY+v9oJL2Le/lDIOpppQnqw1HK702N4LxlD59Vr39OK7e7bdq1qbRtDkcr/O2g3rZ8/WplD5zWZ9Cfv3tM73vteD7w6wMYuQaO7/n3UbNuv1xhQqH8TWuDRglekRwSTCtoKw2eii8g96YTESR7//dLJgs76u+++tN5qoPAq87mVf6+zvrzd9OFA0P1MF4IroRhOVR4ABKKgPr80m2HJWKN768bElpw+LPQjCW1w/ud7oovIYcJH7V7XOV7/a/MY4makeOj5pjd+4pv5ivdFE5VHgIvtO6+x2603oxYjQ2Y6h47fTbrvdK9ab1Qje5L69k1qpwJIv1gfs+PiNnjCBjghdteZfnx2tVAwdt+HPwZh5GTr+j2/79o4UQ8ft+JynPRnoNYGO31ihGBG6as3nP/kAyNBx+EPwkDCvAQbAG72Zl6Hjv6r5xd1yvbHk9KGgB0H4//9JsDCvA17XrWqdWnsy0GsCHb+xQjEi1IoiSJg3RaaNxRJxyh4J8+Nh6LgtHxE66zF0vB/PwbDk9KGgB0H46U/XG11UXgdc8PQlrfOlL61WDB3vRxZLPnR8XBWEcRiVb30I0yab9e///nqjBf7yL7XOX/7lJMaIHBH64BBj8g+z7QMPhmTRRUGrEoRgUZCkhvjtp5t7I0NwDKlKEMuU1BCkX8z2U8/9gINFaUhwrk5LQD54xmO+5u/1eR5GkxSOZ+eTmwOQfrHaf6pHfF1f8hkP0AQFyg9/f3WGivvBZstKwz/5jsSVBlVNXNmafgK5OhURop9rOtOkhtn0JZHEZv+ZvuWzaELDx9UbRNwPHD2L/cg3/C2KY9j9Jm/Ni/RD6BdNVTVfPamnnP4qSlK4OD25WuWqOv2DJB77P+vbUtXPeVRP9Kye168kyf3ZzXVW3qvqaI6HezMiePWEL6OqX/C4QGlIWEhc2RrXqtruEUnKIUI/1ENUV9fWYV6SAjRyvVK9u4yhng9kPkZVy5+UPPvQx22pqjeIuDcghE6+iur998yt0oBXIBDmR/eqr09IUg0hmqpODogSOHBsJ+HBRHVKJAPb3zHNvG0/ge5BB2UHEffGg9A/UN09cuyL2LQfx65Hu6ofSFIMoV/4ChOzlOBMYJLvg+r730gGs396qHd7xn6S3e/d6eEpHu4NB6Gx6vSCQvfFbVe6mKqOSdKM95o/lfUqyS/Ocn0/KIda3wb2E+7+ttZDLNybDkLvVC+JBNtrDy5V35GkFUJTzQ9IPEi9g1ynJIPo2tH6l0q+faGjWnfwcD9Y9Ce+52w00dyA9sDLdEJM7Ru89qKeP1O9jG7QjFze3fpgX+j2Tm/wcG8uCH0Ad5Rnee+BHdIIoT/uYhr5SUquu/sjevs9BznSmyw0x8O9qSB0YrVlKN8uVYPPSSWmOiHxJvEmOo3OSEdJtD+ARSzcmwyv4dEx0repvk4jhF6qHnjEgerLqKXyjercH+aqb/BwbyYI7enuBeJc7JrecAox1RWJR4m30mlUVlr6Y1+o1BUW7s2FVr/H3NG+HWmbSsz02iuudRaVj3rrE7f6EQv3hoLQW703d8Rv9/qWJGUQOtCcxKu0y6MdYAmdmSzpVw46w8O9mfBOj5DnSHdSiMfAZa8Wb6LxSku/KPUVFu5NhV2dI89cd1OIiV55xpVOovG7Zn6R6e9YuDcShD5pbe6o32r9RJI2nOuJZ5zoeTQO9dovrvUQC/eGwlTH6DPWafow01PPONVZNHJ9Xn5xozkW7g2FiV6hz5VO0odjvfCMCz2OxgP0V/ziTB+AhXtD4VxP0OdEz9MH1aTkhZiNRrWfzu4NhZmeos+pzoZjHOsF+lzo8XAMVZIU+EtB//Wf2X9m/5n9Z/YfI2GzwQKEAtoPa1rEKgCbNRK3AR4VSvywpjXYfzbrC6zD9QYIeDscXnat14dUeL4+QgouyTddt4v1ZjWBC4+/98k4PHwX61ZYb1YM2KxFODxciwBMlghfOj98Q64cAN0qPjwUAigl673i8HBVAMockTaLVnJRtSoAOMFRYAXh7X34+ooiItwI54c/WHkAh/OPrBYkWvD8V6z4MWTA4Fv1IbpGyAD/h0GnxBdzmxj3iNAN/IqVh+gaIWEqwjQ7Msm2k6DO9xarxQl1DwQ4m6kQFZL4N1JHL/eYMBHOr1ovU+LebTzmenjDvksyX7ThZtAym/U1K758pKHlkohDQu46DK1C+5dC60dfb3c7t4u4WwpPEcd/atPtXowACWmmQq7y8Bb4HeiCC/qiIqe3E6mS02PA8TkW8mg3JOzvu9gPGbFySnZUXUuSbSVClTmH4CVCP17LzG0DurQtj5z1ArV2O9UYUewe4DQBGDVPfDnlLFFydM6nO/FzyqfXzHFKx/GUz3dDgJA1Rzd3jGAxFJA1FbcOhDrmdplRtmyZG5JtJEJL5ioj8RChztESwRopJ3Fihy5QwfvbFFKqjQaJEs9cUtplwiwdn3IcBOw5urljBIstBdQwrx0ix8yNNXbSgfLkPrB2aHAp2Fax4Y1V5dvjD24NCUao4arJ4KwISJGROIBtVjZmromYnQtmAKkhypQ4vmpT+lXDp9Q/x6cDANQAcOQOtsQYNY2DikdknY2hgsck7gNrW0OImu0SQmvugyWCqRch/UiSTMtLIjHOOaOAgkdhgHZqxCUg6ZuEDVzF8k4aSonG+IUcl9J5kxWza2TqGbSltWcR4BK2dPdqRkA81V3lPLAu6zaQ41ZjTIy5sWoDyrlseubNmoS4zTfMfU6SfECvxbCsjH03oL/jC4Go0YZUQCrRWOJYzs9VbJKmQaGbt6XNiNvw87lN2IH1xtRLFKLGlCg4o4Y7SAVHLtdEUAJI+l4nxmvH+fiW86VZFekhbMHbWST6yvIoSTgWH1fmnf4hQxRCVo8dvRfHULyjYQEC2xocOm/ggfVmTZlRs6nBAOxmm4RJvw1RVgWb2mm57rkzVUKxpKzjgqTnWYquWRNZm2945CYI69LkPscmIIMuK9oq6NEOQVis8yRIgwgIdcmgUrIpbXcOJT0RrMZqCmiBig1vwAfURs3GjOh4SWa/sQGmWmbSHnYqqDc1Qj+BuawrOAibjCCZWRW+29fWG4MhHyuYrPNjyNAlJOM3Xn6kFbAzDrpUP4vqAA04TBkRQllVwbPhhcFKvZbrYDWQUPI4uYDuKmXLruIKbJITNVxkIfznD0R3iakkdqLZspNw+qgpGbCTMExZ3kdtmaJCqgnKreMhUAYitHozIxxjSfuck5PcRZlYhIisFT2PISAzOhaoSz+SLEbx8fEpuqoi6zRVyDJYBa3pS6uqWazBVshzRVpDondV6C56XtuMuEkqJpP0sNahJbc264prIguySACiPdVHnLNWIlwZSUlzi9IHqTB7FVQKGiokm4FqQQM88NrZNzYfzNSJwpqAy7wGkBFTEVmDTQXXCX4AAyFwuHxkIYGGHoM1y942/UOcs1YiXBm5oS/wubK0Ptgx/0IF+UAvToOF4sb8KTJtjT0aMsDqBuTk7RMjLq3exIZLk+jlmuoepuumpnUHOoS9peSqrOEJldwCZMf3qUFah74hlJU9c7UxHak8ZIj/CK5168iPcudwjTZIj/hcBfnQ8at2GJEMHQdVT2e6KyG7UWzAxA2hDEw8MCEqXlsJvuSCctfkF3L+AY0luMeEGxILsEqJ9x2SC/at9lnvuJjF4n3qOONJGxTmYwXwzWjNhz42AnaDiOxmNCJatibrhM7fIKiZ+yW4bKHKTAerVuod2aDiZc6jsuCqA7Ot2qY3G4SY7V8botkUpmcQmHPUSEJjNQ77TAeyatZU3NgXRD4HAlRorSklX9AIu1P+uRzp7e3nPg0XbYd+gCtM3DNCCYyaym6Bt0kL5KwEyKy2hq8N9gFGgllI6FddiCMy4CqhBdKJvsEMOr/UWIO6dIQuCaohYk9q0QXpquUshXxgDrpOO4ioPDCnH8FOeoRrTJCadWcaRyDOgFiEjFQLAJ5nMFgSEs0i9Muz3JEhq7LkwYy9BFQ8MMs76qV2/O96Y9qP4CMlUMKUNYc1Zc6rViXCXXsbBLGpBTiSL9KnaWTkRA8TTWZnVjctEmBskNGxBU3nRkVCqyNVeFNGq1BQI4sXKGSwg7zMAnUhtMgiOo9G5DQBL05YmPfcVMxcjUji4gt/PlWzZQekskhLxMT3utmwqZdiWcg+dy6u+U/SbNkB6SIlESKCK+qKa5I4NrEE2PUZyqy6Z2RNih1xGc456qPvuHHUS+C8jJWn5gq8EdGGly6AJuYaSMoqrkMe+iE8CzP3p2jWwfKGqyyUzvH95D2JY3VTcTmcJ+vd19PYhEsyOP8k1jCbdq9xWzH3I1PrOLAvaG3ZkuSiHZtTUcNtuhxqLBEkQRGuWcjDbTodahxJUtg3aw2lSCf3//Wf2X9m/5n9Z/afVRuO9QL5m9CFHg/HmOkp+pzqDE3ibjV8q+Oj8QA988m+0Jk+AAv3hsK5nqDPiZ6nDzM99YxTnUXjM/R5+cWN5li4NxQm+qTQ50onaBJ3q+FbHR+NQ732i2s9xMK9ofAd+rjQZ6zT9GGiV55xpZNo/K5P5BeZ/o6Fe0Phk34B+tT6KX14jK48Y6U30Xilj8ovSn2FhXsjYSG0q3PUb0Jz3Q3iLW34qTT3jFwPovFU+jl+ca9nWLg3FXb0CHmO9B2ixLrhNYlXwwEzkmiCPuqtP/aFbvUjIu7NhLd6jzz3+jaN+A6z6NXyNCorLX2i1BUW7g2FhdCXMU+oP7dBrKUPL9UchHiTdAeqL6PyRs3xsD+HrvoGC/fmwp7uXqB9E7rY1T1Mifdp4hGT6KerhL4tHfnDSEeIuDcTFkKvdYo4U30dxFka8cdD9AOJJ0n3Qe/+iM5c46r/k9FM6hwL9ybDieol0jehS9UTTIk9Rs9IvEi6M5DrBjlBfOfFCKbQ7Z3eIOJ+oOjJsQy0FCjf9xS093gSex8tD7bxYaPc9GMHOxlX/1LJty90VOsOIu7NhYXQGNywfSHmDqazphMLofeaBy8l6Ul3lut7ksHsH2p9G9hPuPvbWg8RcW8yLITeqV4SsiuF6FLB1I304rf3qh8oyaVShOiD6vvfBuX0UO/2jP0ku9+708NTLNynG/2JToPPQXMD8wuJL6bwdWqxEKKp6uSATG5IWoaw/mu9dDBRnRLJwPZ3VEdz234S3c9HqjuYuE83erPA7vcQKA1J3LWP2xMd7So4H5FiLILovbxTXV1bP8oySQEauV6p3gXvJjHYf4yqlrfJsw993JaqeoOJ+9SjH//Oa9X7o7lVGvAKBML86F719Ulwx5l4P3Sqqvnq6uT0gpIULk5Prla5qk7/iCcyBDTzel9m18/rV5Lk/uzmOivv1XRQMHG/DWJhDg5bVa3HoDQkLPRu3xrXqtruUXBPOxZC9HI606SG2fQlkcRm/83qoyY0fFy9wcR9utGzfqS373YVx7C785ZMLzn9WICu28FjJuez40RxPDuf3PxUZJIuVvtP9er3w/wBSYLyw99fneHiPvXoWYQ+fYcpDUmjd/vW9JOVq9OQEMRpXl+DiMRsPwXdpzZYlIZE52qMWdzW0NcQv/3Uc5/iYFEaEpyrNc2ICalISmfF2X9m/5n9Z/af2X9m/5n9Z/afFXwi/CaVkN+p4pIiUeqUsDURKzIEQETCVYT9SBwlKZ4jzl+55BDi1CK2lA2tVBwSYs12IcJfNLNVZS9ERfhP4VKSIl4j9Jwfvq9+FXO1aTKClRE7Qr9pTKwIXbOHh28Hyig9P2xJoIQlP/pItiBtnoV7khf2UDsvkzhUMJdEJYeG9fqIEtF2OYnHCJUVW6FYkgD22bkiI4mO0Lqqsq3Id8ZHSTLEYH+fQmIEOhJvEeqYi3FNtDaf0oBuC3NLMOSbQMNgtKCi2QqyimuSoQNCIIBCjk5xxNtnoYZs2QNxniI0Yt5k8CAs60EtBLF/RDTQEhkTffB5a57aYQSgsFuFHGhAIBuwrkAx9xLgv88IDuvSmrl1YAlbMneDUNj/7hoPDlMRaeQYsgZGhgxYZd1dyDFgZwa9icZb2pCKp2OuKYwsEBEVWHutAWKPB4eriDJybMf8htthC7Ccl/7h6E/0zlgaMzc2dswNxsaohgZGLTMXm0CcFatZV3EJ9K4DBVWXEdWbQFppKW1A5TZsIfeYJbvPqgoRhS+sBwD0ckqDUFM5x4MBXaAEMoaa+qyByjZQahYoGgYgcDwkNhqPKcMOtsBwhhMBcTKKTgPEwFqsajIiMx7ckGHfnOOlLGCfi4ayjrnlPqd1sDKHt577IQCOENfqjnnpK6NIQ5nO3YiyYoDdgGoSh3yrb7MBcJGR0V3CQVoi3ueOCNRlI4fYF/2EzMzYj4hssZqD3XylHQAQ1k0B6pPItFyQhPTeqOCWXNWvrRdKy4AA5tYtaUgCCHVZwWdMHpwxB9sdylTWoZGQI7RcDECjhCYH0T0EWaCNQAXeFY3VITz/lLgioop7DIWypmsrWCPHSkkyNNikE6p2DCdVYrObdEBGE0Jv4XgYg922huHIG0nvWsVi/saYYB2Ch3hS62nsgp1YMVqMc7gwlII0AgZ8ZoTKLdmIxCnCsRpeWVZkZKIxdpbDEySbIhLXmBQkNjKBhZFk6RszN47dXLXtIg7kiw1nwOKq1WV0SuaSoL66siqmEbPNCEpqYlpNj98YwuD792CsidbjCn4hB0BFXx+oWWbO08jOGaEF8ziQtOw4VnruhzIg8c1alRknjU7WskPfEurLmEtAbUsvCo4JdI/JMAYEvqvTcXrYgYI8yrvefRrZddWq0LorwLd9xrca3bU6ZGUhztPDtpBo8sL1Ob8HwyEJShZbG6VgQyvZTP7PwaD2bjUVaLelNWhtPpEQOtDn+GYtpyRLsqWNVkArmfyfrEXt3Woq0G4r0bCc9O3/y0POVUZbpLg0k8m2WyEVMM/Qa3oXWi63Ri98+vf2K6QAJhr6y2s/POldeNCYt0jvw4//p9uxMEWe4ydGGo57ysMnPQwPb4tqmq/zw79uz8IO6jx8YqjhYVqdCLP/zP4z+8/sP7P/zP4z+8/sP7P/zP4z+8+DoSQA) ### snpe-dlc-diff snpe-dlc-diff compares two DLCs and by default outputs some of the following differences in them in a tabular format: - unique layers between the two DLCs - parameter differences in common layers - differences in dimensions of buffers associated with common layers - weight differences in common layers - output tensor names differences in common layers - unique records between the two DLCs (currently checks for AIP records only) usage: snpe-dlc-diff [-h] -i1 INPUT_DLC_ONE -i2 INPUT_DLC_TWO [-c] [-l] [-p] [-d] [-w] [-o] [-i] [-x] [-s SAVE] required arguments: -i1 INPUT_DLC_ONE, --input_dlc_one INPUT_DLC_ONE path to the first dl container archive -i2 INPUT_DLC_TWO, --input_dlc_two INPUT_DLC_TWO path to the second dl container archive optional arguments: -h, --help show this help message and exit -c, --copyrights compare copyrights between models -l, --layers compare unique layers between models -p, --parameters compare parameter differences between identically named layers -d, --dimensions compare dimension differences between identically named layers -w, --weights compare weight differences between identically named layers. -o, --outputs compare output_tensor name differences names between identically named layers -i, --diff_by_id Overrides the default comparison strategy for diffing 2 models components. By default comparison is made between identically named layers. With this option the models are ordered by id and diff is done in order as long as no more than 1 consecutive layers have different layer types. -x, --hta compare HTA records differences in Models -s SAVE, --save SAVE Save the output to a csv file. Specify a target file path. Copy to clipboard ### snpe-dlc-info snpe-dlc-info outputs layer information from a DLC file, which provides information about the network model. usage: snpe-dlc-info [-h] -i INPUT_DLC [-s SAVE] required arguments: -i INPUT_DLC, --input_dlc INPUT_DLC path to a DLC file optional arguments: -s SAVE, --save SAVE Save the output to a csv file. Specify a target file path. -m, --memory Show detailed information about memory usage. -d, --display_all_encodings Show detailed axis-quantization encoding information. -t, --dump_framework_trace Save framework trace info into the csv file that was passed to --save option. Copy to clipboard ### snpe-dlc-viewer snpe-dlc-viewer visualizes the network structure of a DLC in a web browser. usage: snpe-dlc-viewer [-h] -i INPUT_DLC [-s] required arguments: -i INPUT_DLC, --input_dlc INPUT_DLC Path to a DLC file optional arguments: -s, --save Save HTML file. Specify a file name and/or target save path -h, --help Shows this help message and exits Copy to clipboard Additional details: The DLC viewer tool renders the specified network DLC in HTML format that may be viewed on a web browser. On installations that support a native web browser, a browser instance is opened on which the network is automatically rendered. Users can optionally save the HTML content anywhere on their systems and open on a chosen web browser independently at a later time. - Features: > > > - Graph-based representation of network model with nodes > depicting layers and edges depicting buffer connections. > - Colored legend to indicate layer types. > - Zoom and drag options available for ease of > visualization. > - Tool-tips upon mouse hover to describe detailed layer > parameters. > - Sections showing metadata from DLC records - Supported browsers: > > > - Google Chrome > - Firefox > - Internet Explorer on Windows > - Microsoft Edge Browser on Windows > - Safari on Mac ### snpe-platform-validator DESCRIPTION: ------------ snpe-platform-validator is a tool to check the capabilities of a device. This tool runs on the device, rather than on the host, and requires a few additional files to be pushed to the device besides its own executable. Additional details below. REQUIRED ARGUMENTS: ------------------- --runtime Specify the runtime to validate. : gpu, dsp, aip, all. OPTIONAL ARGUMENTS: ------------------- --coreVersion Query the runtime core descriptor. --libVersion Query the runtime core library API. --testRuntime Run diagnostic tests on the specified runtime. --targetPath The directory to save output on the device. Defaults to /data/local/tmp/platformValidator/output. --debug Turn on verbose logging. --help Show this help message. Copy to clipboard Additional details: - *Files needed to be pushed to device (Please note, we have to push theStub.so and Skel.so of appropriate DSP architecture version, e.g., v68, v73) :* > > > // Android > bin/aarch64-android/snpe-platform-validator > lib/aarch64-android/libSnpeHtpV73CalculatorStub.so > lib/aarch64-android/libSnpeHtpV73Stub.so > lib/hexagon-${DSP_ARCH}/unsigned/libCalculator_skel.so > lib/hexagon-${DSP_ARCH}/unsigned/libSnpeHtpV73Skel.so > > // Windows > bin/aarch64-windows-msvc/snpe-platform-validator.exe > lib/aarch64-windows-msvc/calculator_htp.dll > lib/aarch64-windows-msvc/SnpeHtpV73Stub.dll > lib/hexagon-${DSP_ARCH}/unsigned/libCalculator_skel.so > lib/hexagon-${DSP_ARCH}/unsigned/libSnpeHtpV73Skel.so > Copy to clipboard - example: for pushing aarch64-android variant to /data/local/tmp/platformValidator > > > adb push $SNPE_ROOT/bin/aarch64-android/snpe-platform-validator /data/local/tmp/platformValidator/bin/snpe-platform-validator > adb push $SNPE_ROOT/lib/aarch64-android/ /data/local/tmp/platformValidator/lib > adb push $SNPE_ROOT/lib/dsp /data/local/tmp/platformValidator/dsp > Copy to clipboard ### snpe-platform-validator-py DESCRIPTION: ------------ snpe-platform-validator is a tool to check the capabilities of a device. The output is saved in a CSV file in the "Output" directory, in a csv format. Basic logs are also displayed on the console. REQUIRED ARGUMENTS: ------------------- --runtime Specify the runtime to validate. : gpu, dsp, aip, all. --directory Path to the root of the unpacked SDK directory containing the executable and library files. OPTIONAL ARGUMENTS: ------------------- --buildVariant Specify the build variant (e.g: aarch64-android) to be validated. --deviceId Uses the device for running the adb command. Defaults to first device in the adb devices list. --coreVersion Outputs the version of the runtime that is present on the target. --libVersion Outputs the library version of the runtime that is present on the target. --testRuntime Run diagnostic tests on the specified runtime. --targetPath The path to be used on the device. Defaults to /data/local/tmp/platformValidator NOTE that this directory will be deleted before proceeding with validation. --remoteHost Run on remote host through remote adb server. Defaults to localhost. --debug Set to turn on debug log. Copy to clipboard ### snpe\_bench.py python script snpe\_bench.py runs a DLC neural network and collects benchmark performance information. usage: snpe_bench.py [-h] -c CONFIG_FILE [-o OUTPUT_BASE_DIR_OVERRIDE] [-v DEVICE_ID_OVERRIDE] [-r HOST_NAME] [-a] [-t DEVICE_OS_TYPE_OVERRIDE] [-d] [-s SLEEP] [-b USERBUFFER_MODE] [-p PERFPROFILE] [-l PROFILINGLEVEL] [-json] [-cache] Run the snpe_bench required arguments: -c CONFIG_FILE, --config_file CONFIG_FILE Path to a valid config file Refer to sample config file config_help.json for more detail on how to fill params in config file optional arguments: -o OUTPUT_BASE_DIR_OVERRIDE, --output_base_dir_override OUTPUT_BASE_DIR_OVERRIDE Sets the output base directory. -v DEVICE_ID_OVERRIDE, --device_id_override DEVICE_ID_OVERRIDE Use this device ID instead of the one supplied in config file. Cannot be used with -a -r HOST_NAME, --host_name HOST_NAME Hostname/IP of remote machine to which devices are connected. -a, --run_on_all_connected_devices_override Runs on all connected devices, currently only support 1. Cannot be used with -v -t DEVICE_OS_TYPE_OVERRIDE, --device_os_type_override DEVICE_OS_TYPE_OVERRIDE Specify the target OS type, valid options are ['android-aarch64', 'le', 'le64_gcc4.9', 'le_oe_gcc6.4', 'le64_oe_gcc6.4'] -d, --debug Set to turn on debug log -s SLEEP, --sleep SLEEP Set number of seconds to sleep between runs e.g. 20 seconds -b USERBUFFER_MODE, --userbuffer_mode USERBUFFER_MODE [EXPERIMENTAL] Enable user buffer mode, default to float, can be tf8exact0 -p PERFPROFILE, --perfprofile PERFPROFILE Set the benchmark operating mode (balanced, default, sustained_high_performance, high_performance, power_saver, low_power_saver, high_power_saver, extreme_power_saver, low_balanced, system_settings) -l PROFILINGLEVEL, --profilinglevel PROFILINGLEVEL Set the profiling level mode (off, basic, moderate, detailed). Default is basic. -json, --generate_json Set to produce json output. -cache, --enable_init_cache Enable init caching mode to accelerate the network building process. Defaults to disable. Copy to clipboard ### qairt-dlc-diff qairt-dlc-diff compares two DLCs and by default outputs some of the following differences in them in a tabular format: - unique layers between the two DLCs - parameter differences in common layers - differences in dimensions of buffers associated with common layers - weight differences in common layers - output tensor names differences in common layers - unique records between the two DLCs (currently checks for AIP records only) usage: qairt-dlc-diff [-h] -i1 -i2 [-c] [-l] [-p] [-d] [-w] [-o] [-i] [-s] required arguments: -i1, --input_dlc_one Path to the first dl container archive. -i2, --input_dlc_two Path to the second dl container archive. optional arguments: -c, --compare_copyrights Compare copyrights between models. -l, --compare_layers Compare unique layers between models. -p, --compare_parameters Compare parameter differences between identically named layers. -d, --compare_dimensions Compare dimension differences between identically named layers. -w, --compare_weights Compare weight differences between identically named layers. -o, --compare_outputs Compare output_tensor name differences names between identically named layers. -i, --enable_diff_by_id Overrides the default comparison strategy for diffing 2 models components. By default comparison is made between identically named layers. With this option the models are ordered by id and diff is done in order as long as no more than 1 consecutive layers have different layer types. -s, --save Save the output to a csv file. Specify a target file path. Copy to clipboard ### qairt-dlc-info qairt-dlc-info outputs layer information from a DLC file, which provides information about the network model. usage: qairt-dlc-info [-h] -i [-s] [-m] [-d] [-t] required arguments: -i, --input_dlc Path to a DLC file. optional arguments: -s, --save Save the output to a csv file. Specify a target file path. -m, --display_memory Show detailed information about memory usage. -d, --display_all_encodings Show detailed axis-quantization encoding information. -t, --dump_framework_trace Save framework trace info into the csv file that was passed to --save option. Copy to clipboard Last Published: Oct 02, 2025 [Previous Topic Inference Accuracy](https://docs.qualcomm.com/bundle/publicresource/80-63442-2/topics/accuracy.md) [Next Topic Debug Tools](https://docs.qualcomm.com/bundle/publicresource/80-63442-2/topics/usergroup11.md) Source: [https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html](https://docs.qualcomm.com/doc/80-63442-2/topic/tools.html)