# 4 Network preparation
**Parent Topic:** https://docs.qualcomm.com/doc/80-PT790-993B/topic/dl_inference_tools_part.html
## 4.1 Model preparator
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
The QAic model preparator performs an automated way of generating optimal AIC friendly models for usage. It applies optimizations and performs cleanup for the generated model.
The current version supports TensorFlow and ONNX models. The tool checks the model, applies
shape inference, cleans the model, applies possible optimization, handles the
preprocessing/postprocessing nodes, and generates models as per the given configuration. It
has also capabilities to join compatible models to make one unified model.
The parameters required by the tool are configured in a YAML configuration file for a model and the tool loads this configuration file to execute.
**Parent Topic:** [Network preparation](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
## 4.1.1 Configuration details for Single Model use case
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
The model configuration file defines the parameters required by the tool.
Example:
/opt/qti-aic/tools/qaic-pytools/model_configs/samples/preparator/public/alexnet_model_info.yamlCopy to clipboard
The high-level structure of the model configuration files contains two sections. A description of each of the two sections and their respective fields follows.
**Parent Topic:** [Model preparator](https://docs.qualcomm.com/doc/80-PT790-993B/topic/Introduction_modelprep.html)
## 4.1.1.1 Configuration field details
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
The `INFO.` section is used to define model related parameters.
Table : Model preparator configuration: INFO
| Field Name | Description | Type |
| --- | --- | --- |
| DESCRIPTION | Description of the model. | (Not Mandatory)
String |
| NAME | Name of the model.
It should be **unique** for Model Chaining use case. | **Optional** for Single model use case.
**Mandatory** for Model Chaining use case. |
| MODEL\_TYPE | Category of the model. Supported types are:
['CLASSIFICATION', 'YOLO',
'YOLOV2', 'YOLOV3', 'YOLOV4', 'YOLOV5', 'YOLOV7', 'YOLOV7_TINY', 'YOLOX', 'SSDVGG', 'SSDMOBILENET',
'SSDINCEPTION', 'SSDRESNET', 'EFFICIENTDET', 'RETINANET', 'BERT', 'ROBERTA',
'ALBERTA', 'DEBERTA', 'GPT', 'GPT2', 'GPT3', 'BIGBIRD', 'LONGFORMER',
'TRANSFORMERS', 'FASTERTRANSFORMER', 'SRGAN', 'UNKNOWN', 'CUSTOM_OP'] Copy to clipboard | **Mandatory**
String |
| MODEL\_PATH | Path to the model file.
ONNX: Path to the complete model file (.onnx)
TF:
- Saved Model: Provide folder path containing the “saved_model.pb”, “variables”,
and “assets”
- Frozen Graph: Provide complete model file (.pb)
| **Mandatory for TF/ONNX models. Not required for MODEL\_TYPE==“CUSTOM\_OP”**
String |
| INPUT\_INFO | A list containing the model’s input details. It should be the model’s input
node name and the corresponding shape of that input node. | **Mandatory**
List of List [[]] |
| EXPORT\_TYPE | Default set to “ONNX” for all ONNX Models.
For TensorFlow, if the end model is expected to be in TF, then provide
“TENSORFLOW”, else set to “ONNX” to convert the TensorFlow models to ONNX.
For MODEL\_TYPE ==CUSTOM\_OP, this will represent the type of CUSTOM\_OP model to be
generated. | **Mandatory**
String |
| OPSET | Opset version to be used when converting TF model to ONNX model.
Same is used to generate ONNX model corresponding to CUSTOM\_OP. | **Optional**
Int
Default: 13 |
| DYNAMIC\_INFO | A list containing the model’s input details, which needs to be made dynamic. It
should be the model’s input node name and corresponding dynamic shape of that input
node. | **Mandatory**
List of List [[]] |
| VALIDATE | A Boolean flag that indicates whether to check the outputs of the generated
model on AIC against the native framework. | **Optional**
Boolean
Default: True |
| AIC\_DEVICE\_ID | Device ID to be used for Validation purpose. | **Optional**
Int
Default: 0 |
| WORKSPACE | A directory to save log files, intermediate outputs, and the final
model. | **Optional**
String
Default: ./workspace |
| VERBOSE | Set the INFO/DEBUG/ERROR flags to get the relevant logging and debugging
information. | **Optional**
String
Default: “INFO” |
| INPUT\_LIST\_FILE | Path of the input list file which contains paths for all the inputs. This will
be used for validating the model. | **Optional**
String
Default: None |
| CUSTOM\_OP\_INFO\_FILEPATH | Path of the custom op config file. | **Mandatory** for models having custom ops or MODEL\_TYPE==CUSTOM\_OP.
**Optional** for rest of the model.
Default: None |
The `ATTRIBUTE` section is used to define attributes for
MODEL\_TYPE==CUSTOM\_OP case. This is required for model chaining use case. The values in this
section shall be added as a key value pair. A sample snapshot for this section is added
below.
The PRE\_POST\_HANDLE section is used
to define parameters related to preprocessing and postprocessing of the model. Based on
these parameters, various cleanup passes are applied to modify the preprocessing and
postprocessing nodes.
Table : Model preparator configuration: PRE_POST_HANDLE
| Field Name | Description | Type |
| --- | --- | --- |
| POST\_PLUGIN | Postprocessing plugin to be applied to the model.
**(Applicable only for object detection (OD) models)**
Supported types: `['SMARTNMS', 'QDETECT',
'NONE']` | **Optional**
String
Default: “NONE” |
| PRE\_PLUGIN | A Boolean flag that indicates whether to apply any preprocessing plug-in or
not. | **Optional**
Boolean
Default: False |
| ANCHOR\_BIN\_FILE | Provide the path to anchor files required for the ABP part in the OD models. If
none provided for the OD models, it would try to extract the anchors available in
the model. For external anchors, the file shall be of .bin or .raw extension. For
Yolo based Object Detection models, the file shall contain total\_number\_of\_anchors \*
2 floating point values made by flattening [total\_number\_of\_anchors, 2] array. For
SSD / EfficientDet based Object Detection models, the file shall contain
total\_number\_of\_anchors \* 4 floating point values made by flattening
[total\_number\_of\_anchors, 4] array. | **Optional**
String
Default: None |
| COMPRESSED\_MASK | Generate Compressed Mask variant of the model. (**Applicable only for NLP models with attention mask as an input.)**
With this flag, the attention mask computation path is replaced with efficient
implementation of MaskedSoftmax CustomOp.
Limitation: Only Default value (False) is supported in this release. | **Optional**
Boolean
Default: False |
| TRANSFORMER\_PACKED\_MODEL | Flag indicating whether to generate packed variant of the model. **(Applicable only for NLP models with attention mask as an input.)**
Limitation: Only Default value (False) is supported in this release. | **Optional**
Boolean
Default: False |
| NMS\_PARAMS | Section to define various NMS related parameters **applicable for object detection models only**. | **Optional**
Default: None |
Table : Model preparator configuration: NMS_PARAMS
| Field Name | Description | Type |
| --- | --- | --- |
| MAX\_OUTPUT\_SIZE\_PER\_CLASS | Maximum number of boxes to be selected by non-max suppression per class. | **Optional**
Integer
Default: 100 |
| MAX\_TOTAL\_SIZE | Maximum number of boxes to output from the model. | **Optional**
Integer
Default: 500 |
| IOU\_THRESHOLD | IoU threshold for NMS-related computation. | **Optional**
Float
Default: 0.5 |
| SCORE\_THRESHOLD | Score threshold for NMS-related computation. Boxes whose score value is less
than this will be discarded. | **Optional**
Float
Default: 0.01 |
| CLIP\_BOXES | A Boolean flag that indicates whether to clip the resultant boxes between [0,1]
or not. | **Optional**
Boolean
Default: False |
| PAD\_PER\_CLASS | A Boolean flag that indicates whether to pad the resultant boxes.
If false, the resultant boxes are padded to max\_total\_size.
If true, the resultant boxes are padded to be of length max\_output\_size\_per\_class
\* num\_classes, unless it exceeds max\_total\_size, in which case it is clipped to
max\_total\_size. | **Optional**
Boolean
Default: False |
**Parent Topic:** [Configuration details for Single Model use case](https://docs.qualcomm.com/doc/80-PT790-993B/topic/configuration-details-modelprep.html)
## 4.1.2 Configuration details for Model Chaining use case
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
The model configuration file defines the parameters required by the tool.
Example:
/opt/qti-aic/tools/qaic-pytools/model_configs/samples/preparator/public/ model_chaining_sample_config.yamlCopy to clipboard
The high-level structure of the model configuration files contains two sections. A
description of each of the two sections and their respective fields follows.
**Parent Topic:** [Model preparator](https://docs.qualcomm.com/doc/80-PT790-993B/topic/Introduction_modelprep.html)
## 4.1.2.1 Configuration field details
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
The `CHAINING_INFO` section is used to define parameters related to
chaining of different models. Based on these parameters, different models present in
the config will be chained together to form a single unified model.
Table : Model preparator configuration: CHAINING_INFO
| Field Name | Description | Type |
| --- | --- | --- |
| EDGES | It contains which output of a given model is to be connected to
which input of some other model.
**(Applicable only for model chaining use case.)**
e.g.
[
[model\_name\_1::model\_1\_output\_1,
model\_name\_2::model\_2\_input\_1],
[model\_name\_2::model\_2\_output\_1,
model\_name\_3::model\_3\_input\_1],
]
This would indicate that the model\_1’s output\_1 will be connected
to model\_2’s input\_1 and same way model\_2’s output\_1 will be
connected to model\_3’s input\_1. | **Mandatory for Model Chaining use case.** |
| WORKSPACE | A directory to save log files, intermediate outputs, and the
final model. | **Optional**
String
Default: ./workspace |
| EXPORT\_TYPE | Export type of the final model. It can be “ONNX” or
“TENSORFLOW”. | **Optional**
String
Default: “ONNX” |
| OPSET | Opset version to be used when generating final chained
model. | **Optional**
Integer
Default: 13 |
| AIC\_DEVICE\_ID | Device ID to be used for Validation purpose. | **Optional**
Integer
Default: 0 |
The `MODELS` section is used to define parameters related to different
models which is to be chained together. The MODELS section will contain different
configuration for each model in the format of single MO`DEL` config.
A sample chained config is present at :
/opt/qti-aic/tools/qaic-pytools/model_configs/samples/preparator/public/
model_chaining_sample_config.yamlCopy to clipboard
**Parent Topic:** [Configuration details for Model Chaining use case](https://docs.qualcomm.com/doc/80-PT790-993B/topic/configuration-details-modelchaining.html)
## 4.1.3 Usage
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Steps to use the model preparator tool:
1. The user must prepare a model preparator configuration file using the structure defined
in [Configuration details for Single Model use case](https://docs.qualcomm.com/doc/80-PT790-993B/topic/configuration-details-modelprep.html) and [Configuration details for Model Chaining use case](https://docs.qualcomm.com/doc/80-PT790-993B/topic/configuration-details-modelchaining.html). Use the sample
configuration as a template. The sample configuration is available at:
/opt/qti-aic/tools/qaic-pytools/model_configs/samples/preparator/public/alexnet_model_info.yamlCopy to clipboard
Sample execution command (refer to the help menu in [Figure : 1](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)):
python qaic-model-preparator.py -help Copy to clipboard
Figure : Model preparator tool help menu
2. The model preparator tool will apply various phases of operations on the model based on the configuration. At the end of the execution, it will provide a summary of the model before the preparator execution and after the preparator execution. It will also provide details of the various passes that it executed. The screenshot of the same for a YoloV5 model with a post\_plugin value of SMARTNMS is provided below. The configuration for this can be found at:
/opt/qti-aic/tools/qaic-pytools/model_configs/samples/preparator/public/yolov5_ultralytics_model_info_smart_nms.yaml Copy to clipboard
Sample execution command:
python -W ignore qaic-model-preparator.py --config /opt/qti-aic/tools/qaic-pytools/model_configs/samples/preparator/public/yolov5_ultralytics_model_info_smart_nms.yaml Copy to clipboard
Figure : Model preparator tool before summary
Figure : Model preparator tool after summary
Note: The model has a fewer number of operators due to the optimizations performed by the model preparator tool. Model summary before and after will provide an overview of model details like number of parameters, operator count, opset version for ONNX models, input and output information (shape, Dtype, and names), and also various stage level information of preparator toolchain.
**Parent Topic:** [Model preparator](https://docs.qualcomm.com/doc/80-PT790-993B/topic/Introduction_modelprep.html)
## 4.1.4 Limitations
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
The following are some known issues/limitations of the model preparator tool:
1. Currently the tool supports ONNX and TensorFlow (TF) models.
2. The recommended way of working with failed TF models is to enable “EXPORT\_TYPE” to “ONNX”, convert the TF model to ONNX, and generate an optimal AIC100 model.
3. Not all TF models can be converted to ONNX seamlessly. In case of TF2ONNX conversion failures, check the same in TF2ONNX repository.
4. Failure of shape inference in resize operator (batch symbol not being propagated as expected).
5. For model chaining use case, MODEL\_TYPE==CUSTOM\_OP is not supported. So Interconnecting
models with single custom operation is not possible. However, user can still interconnect
onnx models regardless of any model having custom ops or not.
6. For model chaining use case, EXPORT\_TYPE==TENSORFLOW is not supported.
7. Models with Quantize-Dequantize nodes are not supported.
8. The tool is verified and tested on opsets 11-13 for Onnx models.
9. Currently, for the yaml file, TRANSFORMER\_PACKED\_MODEL = False and COMPRESSED\_MASK =
False is supported.
**Parent Topic:** [Model preparator](https://docs.qualcomm.com/doc/80-PT790-993B/topic/Introduction_modelprep.html)
## 4.1.5 Debugging strategy
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Enable the debug flag “VERBOSE” to print much of the debug logs. The supported flags in “VERBOSE” mode are: ['INFO', 'DEBUG', 'WARNING', 'TRACE'].
## Summary stages
| Stage | Description |
| --- | --- |
| Native Checker | It is checking the validity of the provided model. |
| Shape Inference | It infers and adds shape information in the model as per the input shape provided in config. |
| Model Simplifier Optimization | It applies the various graph optimizations and graph cleanup on the model. |
| Tensorflow Optimization | For TF models, it applies a set of graph optimizations and cleanup. |
| TF2Onnx model Conversion | For TF models whose export type is ONNX, this stage converts such TF models into ONNX. |
| Pattern Optimizer | Based on provided model type, various type of patterns can be identified to apply architecture-specific modifications. |
| Dynamic model generation | Static model can be converted dynamic model based on provided config. |
| Post Process Handler | Object detection models can be converted into SMARTNMS variant or QDETECT variant based on provided config. |
| Validation Checker | Checks the final model by running an inference using random data. |
## FAQs
- Not sure which MODEL\_TYPE to select?
- Use UNKNOWN if MODEL\_TYPE is not known or not available in the list of model types.
- If the model is NLP model or has Transformers based architecture, then user shall
use TRANSFORMER as model type to apply transformer based optimizations on top of
general optimizations.
- Which one to select from SMARTNMS and QDETECT?
- These options are only applicable for the object detection model. If the model’s postprocessing routine is required to be run on the host (CPU), then select SMARTNMS. If it is required to run on AIC card, then select QDETECT. For non-object detection models, use NONE.
- In case of No space left on device errors, please change the temporary directory
by calling export TMPDIR=’./tmp\_dir/’.
**Parent Topic:** [Model preparator](https://docs.qualcomm.com/doc/80-PT790-993B/topic/Introduction_modelprep.html)
## 4.2 TF optimizer
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
TensorFlow graphs typically have many dynamic tensor and control flow ops. Many of these ops can be removed by applying different optimizers over the TF graph. This TF optimizer tool removes such redundant dynamic tensor and control flow ops and performs several optimizations to yield a highly simplified graph to be fed to the AIC compiler.
**Parent Topic:** [Network preparation](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
## 4.2.1 Integration with qaic-prepare-model
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
The TF optimizer tool is integrated into the qaic-prepare-model script (refer to [YAML model profiles](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-profiles.html) for information on qaic-prepare-model.py). The requirements and installation are the same as “qaic-prepare-model".
The YAML file to be provided is the same as qaic-prepare model.
###################################################################
# Sample YAML file
##################################################################
version: "1.1"
name: "ResNet-50 ONNX"
download:
url: https://s3.amazonaws.com/download.onnx/models/opset_6/resnet50.tar.gz
model-file: "./resnet50/model.onnx"
framework: "ONNX"
model-inputs:
- node: "gpu_0/data_0"
dtype: "float32"
shape: [ "*", 3, 224, 224 ]
model-outputs:
- node: "gpu_0/softmax_1"
qaic-compiler:
- name: "Generic"
cores: 1
instances: 1
- name: "PCIe HHHL 2.0"
bs: 1
cores: 1
ols: 1
mos: 1
instances: 14
num-iter: 5000
convert-to-quantize:
quantization-schema-activations: "symmetric_with_uint8"
quantization-schema-constants: "symmetric_with_uint8"
quantization-precision: "Int8"
quantization-precision-bias: "Int32" Copy to clipboard
**Parent Topic:** [TF optimizer](https://docs.qualcomm.com/doc/80-PT790-993B/topic/tf_optimizer_introduction.html)
## 4.2.2 Example snippet of running TF optimizer
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Command: `python -W ignore qaic-prepare-model.py --profile example_iupg.yaml`
Figure : TF optimizer run
It provides the following information:
1. Replacing input placeholders with fixed input shape.
2. Summary of the model: Inputs and Outputs.
3. Summary of the optimizers.
**Parent Topic:** [TF optimizer](https://docs.qualcomm.com/doc/80-PT790-993B/topic/tf_optimizer_introduction.html)
## 4.2.3 Optimization details
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
The TF optimizer looks for control flow operators inside the model and replaces them with the appropriate/relevant operators.
Major optimizations include:
1. Switch-merge optimizer:
This arises due to the “tf.while\_loop()” and “tf.cond()” APIs. The “prediction” flag of the switch operator is evaluated, and the corresponding path is selected for the model. A switch operator whose prediction flag is not constant cannot be optimized.
2. Loop unrolling:
This arises due to the “tf.while\_loop()” APIs. The loop is unrolled to make a directed acyclic graph (DAG) based on the number of loop iterations. It cannot be applied in cases where the number of loop iterations is not fixed or there is a nested loop.
3. TensorArray optimizations:
This includes all tensor-array operators like TensorArray, TensorArrayWrite, TensorArrayRead, TensorArrayGather, TensorArrayScatter, and so on. This optimization replaces tensor-array operators with relevant known and supported operators.
**Parent Topic:** [TF optimizer](https://docs.qualcomm.com/doc/80-PT790-993B/topic/tf_optimizer_introduction.html)
## 4.2.4 TF optimizer in action
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Figure : Removal of switch-merge and loop operators
**Parent Topic:** [TF optimizer](https://docs.qualcomm.com/doc/80-PT790-993B/topic/tf_optimizer_introduction.html)
## 4.2.5 Limitations
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
1. The optimizer is not tested for nested loops.
2. TensorArray operators cannot be replaced if the size of the TensorArray operator is dynamic.
3. Loop optimizer does not work if the number of iterations is dynamic.
4. Switch optimizer works if the “Pred” flag of the switch node is constant.
**Parent Topic:** [TF optimizer](https://docs.qualcomm.com/doc/80-PT790-993B/topic/tf_optimizer_introduction.html)
## 4.3 YAML model profiles
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
QAic model profile is a new deliverable for the model onboarding and tuning workflows. This is a YAML file that captures model details such as download URL, input/output node information, and optimal compiler flags (ols, mos, and so on).
A companion Python script ‘`qaic-prepare-model.py` ’ will download the model and perform common fixups such as freeze graph, optimize for inferencing, fix input shape, add shape info, and so forth, making it ready to use with qaic-exec.
**Parent Topic:** [Network preparation](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
## 4.3.1 Prerequisites
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
$ cd /opt/qti-aic/examples/apps/qaic-prepare-model
$ pip3 install -r requirements.txt
Copy to clipboard
The required ONNX and TensorFlow Python packages might be incompatible with your
environment. Therefore, it is recommended to run qaic-prepare-model in a virtual Python
environment:
# virtualenv(optional)
$ pip3 install virtualenv
$ virtualenv venv
$ source venv/bin/activate
$ pip3 install -r requirements.txt
Copy to clipboard
To exit the virtual Python environment:
$ deactivateCopy to clipboard
**Parent Topic:** [YAML model profiles](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-profiles.html)
## 4.3.2 Sample usage
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Prepare model file:
$ python3 qaic-prepare-model.py --profile example.yaml Copy to clipboard
Prepare model file, compile, and run network:
$ python3 qaic-prepare-model.py --profile example.yaml --run 'PCIe HHHL 2.0'Copy to clipboard
**Parent Topic:** [YAML model profiles](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-profiles.html)
## 4.3.3 Command line options
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
usage: qaic-prepare-model.py [-h] --profile PROFILE [--output OUTPUT_DIR]
[--compile COMPILE_CONFIG] [--run RUN_CONFIG]
[--device-id DEVICE_ID]
[--aic-hw-version AIC_HW_VERSION] [--no-fixup]
[--print-ops] [--input-list-file INPUT_LIST_FILE]Copy to clipboard
Table : Command line options
| Option | Description |
| --- | --- |
| -h, --help | Show this help message and exit. |
| --profile PROFILE | Filename of model profile. |
| --output OUTPUT\_DIR | Working directory for output files. |
| --compile COMPILE\_CONFIG | Compile the named model configuration. |
| --run RUN\_CONFIG | Compile and run the named model configuration. |
| --device-id DEVICE\_ID | AIC100 device ID, default 0. |
| --aic-hw-version AIC\_HW\_VERSION | Hardware version of AIC100, default 2.0. |
| --no-fixup | Disable all model optimizations and shape modifications. |
| --print-ops | Prints operators and layers present in the model. |
| --input-list-file INPUT\_LIST\_FILE | Name of file (.txt) containing list of inputs (one line per inputs). For example: If the model has one input, then one entry per line. If a model has more than one input, then each line needs to be comma (,) separated. The order of inputs should be same as model input order. |
**Parent Topic:** [YAML model profiles](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-profiles.html)
## 4.3.4 Model details
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
**Parent Topic:** [YAML model profiles](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-profiles.html)
## 4.3.4.1 PyTorch
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
- Model download, compile, and execution.
**Parent Topic:** [Model details](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-preparation.html)
## 4.3.4.2 Caffe2
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
- Model download, compile, and execution.
**Parent Topic:** [Model details](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-preparation.html)
## 4.3.4.3 ONNX
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
- Model download, compile, and execution.
- Supports ONNX v1.8.0.
- Strip doc\_strings.
- Set static input shape information.
- Simplify model with onnx-simplifier to replace redundant operators with constant outputs. onnx-simplifier is preinstalled via the Python package installer (pip). More details at [https://github.com/daquexian/onnx-simplifier](https://github.com/daquexian/onnx-simplifier).
- Optimize model with standard passes of the ONNX Optimizer Python API. More details at [https://github.com/onnx/onnx/blob/master/docs/Optimizer.md](https://github.com/onnx/onnx/blob/master/docs/Optimizer.md).
- Infer shape information.
- In the case of external data support (.data), when creating the model with ONNX API's convert\_model\_to\_external\_data or onnx.save\_model, make sure the location argument is not given as an absolute path.
**Parent Topic:** [Model details](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-preparation.html)
## 4.3.4.4 TensorFlow
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
- Supports TensorFlow >= 1.14.
- Remove training nodes.
- Set static input shape information.
- Add shape information to graph.
- Runs TF optimizer [refer to [TF optimizer](https://docs.qualcomm.com/doc/80-PT790-993B/topic/tf_optimizer_introduction.html)] to remove loops and redundant nodes and output a simplified graph.
**Parent Topic:** [Model details](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-preparation.html)
## 4.3.5 QAic model profile reference
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
**Parent Topic:** [YAML model profiles](https://docs.qualcomm.com/doc/80-PT790-993B/topic/yaml-model-profiles.html)
## 4.3.5.1 Information
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Table : Command line options
| Option | Description |
| --- | --- |
| Version | Compatible SDK version. |
| Name | Friendly name of the model. |
| url | URL of the model. If the URL points to a .tar.gz or .tgz archive, the
contents are automatically extracted. |
| Model-file | Local filename of the model. |
version: "1.2"
name: "MobileNet V2 ONNX"
download:
url: "https://github.com/onnx/models/raw/master/vision/classification/mobilenet/model/
mobilenetv2-7.onnx"
model-file: "mobilenetv2-7.onnx"Copy to clipboard
**Parent Topic:** [QAic model profile reference](https://docs.qualcomm.com/doc/80-PT790-993B/topic/qaic-model-profile-reference.html)
## 4.3.5.2 Framework
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Table : Command line options
| Option | Description |
| --- | --- |
| Name | Framework name. Options are "Caffe2", "ONNX", or "TensorFlow". |
| remove-training-nodes | Remove training nodes from the graph. Values: true or false |
| onnx-simplify | Optimize the graph with ONNX simplifier. Values: true or false |
framework:
name: "ONNX"
onnx-simplify: trueCopy to clipboard
A shortened version is also available when no graph optimizations are required:
framework: "ONNX"Copy to clipboard
**Parent Topic:** [QAic model profile reference](https://docs.qualcomm.com/doc/80-PT790-993B/topic/qaic-model-profile-reference.html)
## 4.3.5.3 Model inputs
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Table : Command line options
| Option | Description |
| --- | --- |
| node | Name of the input node. |
| dtype | Data type of the node. Options are "float32", "float16", "int32", "int64", "bool". |
| shape | Shape dimensions. If a dimension is marked as "\*", "?", or -1 then it is automatically replaced with the batch size. |
model-inputs:
- node: "data"
dtype: "float32"
shape: [ "*", 3, 224, 224 ] Copy to clipboard
Randomized data is generated for the inputs when compiling and running the model.
**Parent Topic:** [QAic model profile reference](https://docs.qualcomm.com/doc/80-PT790-993B/topic/qaic-model-profile-reference.html)
## 4.3.5.4 Model outputs
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Table : Command line options
| Option | Description |
| --- | --- |
| Node | Name of the output node. |
model-outputs:
- node: "mobilenetv20_output_flatten0_reshape0"
- node: "output_2"Copy to clipboard
**Parent Topic:** [QAic model profile reference](https://docs.qualcomm.com/doc/80-PT790-993B/topic/qaic-model-profile-reference.html)
## 4.3.5.5 QAic-compiler
Source: [https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html](https://docs.qualcomm.com/doc/80-PT790-993B/topic/network-preparation.html)
Table : Command line options
| Option | Description |
| --- | --- |
| node | Name of the output node. |
| bs | Batch size. |
| cores | Number of NSP compute cores. |
| ols | Overlap split factor. |
| mos | Maximum output channel split. |
| instances | Number of instances of the model to run. |
| minimize-host-traffic | When set to true, reduces pre-post processing of AIC100 on host. |
| convert-to-fp16 | When set to true, runs all floating-point computation in fp16 precision. |
| convert-to-quantize | Enables quantization. |
| quantization-schema-activations | Quantization schema to use for activations. Options are "asymmetric","symmetric", "symmetric\_with\_uint8" (default), and "symmetric\_with\_power2\_scale". |
| quantization-schema-constants | Quantization schema to use for constants. Options are "asymmetric","symmetric", "symmetric\_with\_uint8" (default), and "symmetric\_with\_power2\_scale". |
| quantization-precision | Quantization precision to use. Options are "Int8" (default) and "Int16". |
| quantization-precision-bias | Quantization precision bias to use. Options are "Int8" and "Int32 (default)". |
qaic-compiler:
- name: "PCIe HHHL 2.0"
bs: 8
cores: 4
ols: 2
mos: 1
instances: 3
minimize-host-traffic: true
num-iter: 5000
convert-to-quantize:
quantization-schema-activations: "symmetric_with_uint8"
quantization-schema-constants: "symmetric_with_uint8"
quantization-precision: "Int8"
quantization-precision-bias: "Int32" Copy to clipboard
**Parent Topic:** [QAic model profile reference](https://docs.qualcomm.com/doc/80-PT790-993B/topic/qaic-model-profile-reference.html)
Last Published: Jul 26, 2023
[Previous Topic
QAic config utility](https://docs.qualcomm.com/bundle/publicresource/80-PT790-993B/topics/Card-Health-Resources-and-Logs.md#Card-Health-Resources-and-Logs_qaic_config_utility) [Next Topic
Network compilation](https://docs.qualcomm.com/bundle/publicresource/80-PT790-993B/topics/network-compilation.md)