# CNN to QNN for Windows Host on Linux Target

Note

This is **Part 2** of the CNN to QNN tutorial for Windows host machines. If you have not completed Part 1, please do so here.

## Step 3: Model Build on Windows Host for Linux Target

Once the CNN model has been converted into QNN format, the next step is to build it so it can run on the target device’s operating system with `qnn-model-lib-generator`.

Based on the operating system and architecture of your target device, choose one of the following build instructions.

Warning

For cases where the “host machine” and “target device” are the same, you will need to adapt the steps to handle files locally instead of transferring them to a remote device.

Note

Please continue to use the same terminal you were using on your host machine from part 1.

1. Create a directory on your host machine where your newly built files will live by running:

mkdir -p /tmp/qnn_tmp
        Copy to clipboard
2. Navigate to the new directory:

cd /tmp/qnn_tmp
        Copy to clipboard
3. Copy over the QNN `.cpp` and `.bin` model files:

Copy-Item -Path "$env:QNN_SDK_ROOT\examples\Models\InceptionV3\model\Inception_v3.cpp", "$env:QNN_SDK_ROOT\examples\Models\InceptionV3\model\Inception_v3.bin" -Destination "C:\tmp\qnn_tmp"
        Copy to clipboard
4. Choose the most relevant supported target architecture from the following list:

Warning

If you don’t know which one to choose, you can run the following commands on your target device to get more information: `uname -a`, `cat /etc/os-release`, and `gcc --version`.

    - For 64-bit Linux targets: `x86_64-linux-clang`
    - For 64-bit Android devices: `aarch64-android`
    - For Qualcomm’s QNX OS: `aarch64-qnx` (Note: This architecture is not supported by default in the QNN SDK.)
    - For OpenEmbedded Linux (GCC 11.2): `aarch64-oe-linux-gcc11.2`
    - For OpenEmbedded Linux (GCC 9.3): `aarch64-oe-linux-gcc9.3`
    - For OpenEmbedded Linux (GCC 8.2): `aarch64-oe-linux-gcc8.2`
    - For Ubuntu Linux (GCC 9.4): `aarch64-ubuntu-gcc9.4`
    - For Ubuntu Linux (GCC 7.5): `aarch64-ubuntu-gcc7.5`
5. On your host machine, set the target architecture of your target device by setting `QNN_TARGET_ARCH` to your device’s target architecture:

$QNN_TARGET_ARCH="your-target-architecture-from-above"
        Copy to clipboard

    For example:

$QNN_TARGET_ARCH="x86_64-linux-clang"
        Copy to clipboard
6. Run the following command to generate the model library, updating the `t` value with the target architecture:

python "${QNN_SDK_ROOT}/bin/x86_64-linux-clang/qnn-model-lib-generator" `
            -c "Inception_v3.cpp" `
            -b "Inception_v3.bin" `
            -o model_libs `
            -t ${QNN_TARGET_ARCH}
        Copy to clipboard

    - `c` - This indicates the path to the `.cpp` QNN model file.
    - `b` - This indicates the path to the `.bin` QNN model file. (`b` is optional, but at runtime, the `.cpp` file could fail if it needs the `.bin` file, so it is recommended).
    - `o` - The path to the output folder.
    - `t` - Indicate which architecture to build for.
7. Run `ls /tmp/qnn_tmp/model_libs/${QNN_TARGET_ARCH}` and verify that the output file `libInception_v3.so` is inside.
- You will use the `libInception_v3.so` file on the target device to execute inferences.
- The output `.so` file will be located in the `model_libs` directory, named according to the target architecture.

> 
> 
> - For example: `model_libs/x64/Inception_v3.so` or `model_libs/aarch64/Inception_v3.so`.

## Step 4: Use the built model on specific processors

Now that you have an executable version of your model, the next step is to transfer the built model and all necessary files to the target processor, then to run inferences on it.

1. Install all necessary dependencies from Setup.
2. Follow the below SSH setup instructions.
3. Follow the instructions for each specific processor you want to run your model on.

**Sub-Step 1: If you haven’t already, ensure that you follow the processor-specific Setup instructions for your host machine :doc:`here </general/setup>`.**

**Sub-Step 2: Set up SSH on the target device.**

> 
> 
> Warning
> 
> 
> Here we use `OpenSSH` to copy files with `scp` later on and run scripts on the target device via `ssh`. If that does not work for your target device, feel free to use any other method of transferring the files over. (Ex. `adb` for android debugging or USB with manual terminal commands on the target device)
> 
> 1. - Ensure that both the host device and the target device are on the same network for this setup.
>     - - Otherwise, `OpenSSH` requires port-forwarding to connect.
> 2. - On your target device, install `OpenSSH Server` if it is not already.
>     - - Ex. For an Ubuntu device, this would look like:
> 
> 
> 
> # Update package lists
>     sudo apt update
>     
>     # Install OpenSSH server
>     sudo apt install openssh-server
>     
>     # Check SSH service status
>     sudo systemctl status ssh
>     
>     # Start SSH service if it's not running
>     sudo systemctl start ssh
>     
>     # Enable SSH service to start on boot
>     sudo systemctl enable ssh
>     Copy to clipboard
> 
> 
> Note
> 
> 
> You can turn off the OpenSSH Server service later by running `sudo systemctl stop ssh` if you want to.
> 
> 3. On your target device, run `ifconfig` to get the IP address of your target device.
> 4. On your host machine, set a console variable for your target device’s `inet addr` address from above (replacing `127.0.0.1` below).
> 
> 
> 
> $TARGET_IP="127.0.0.1"
>     Copy to clipboard
> 
> 5. Also set the username of the desired account on your target device (you can find it by running `whoami` on your target device if you are logged into the desired account).
> 
> 
> 
> $TARGET_USER="your-linux-account-username"
>     Copy to clipboard
> 
> 
> 6. On your host machine, install `OpenSSH Client` by:
> - Opening a Powershell as an administrator.
> - Installing by running `Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0`
> - Verifying the installation by running `Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH.Client*'`
> 
> 
> From this point on you should be able to `ssh` from powershell. You may need to open another PowerShell to do though.

**Sub-Step 3: Follow the steps below for whichever processor you would like to run your model on.**

### CPU

#### Transferring over all relevant files

1. On the target device, open a terminal and make a destination folder by running:

mount -o remount,rw /
        mkdir -p /data/local/tmp
        cd /data/local/tmp
        ln -s /etc/ /data/local/tmp
        chmod -R 777 /data/local/tmp
        mkdir -p /data/local/tmp/inception_v3
        Copy to clipboard
2. On the host device, use `scp` to transfer `libQnnCpu.so` from your host machine to `/data/local/tmp/inception_v3` on the target device.

scp "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnCpu.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
3. Use `scp` to transfer the example built model.
- Update the `x64` folder below to the proper folder for your built model. The folder name depends on your host machine’s architecture.

scp "/tmp/qnn_tmp/model_libs/x64/libInception_v3.so"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
4. Transfer the input data, input list, and script from the QNN SDK examples folder into `/data/local/tmp/inception_v3` on the target device using `scp` in a similar way:

scp -r "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/target_raw_list.txt"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/imagenet_slim_labels"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/scripts/show_inceptionv3_classifications.py"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
5. Transfer `qnn-net-run` from `$QNN_SDK_ROOT/bin/$QNN_TARGET_ARCH/qnn-net-run` to `/data/local/tmp/inception_v3` on the target device:

scp "$QNN_SDK_ROOT/bin/$QNN_TARGET_ARCH/qnn-net-run" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard

#### Doing inferences on the target device processor

1. Open a PowerShell instance on the target device.
- Alternatively, you can `ssh` from your host machine, run the following command to `ssh` into your target device.
- These console variables were set in the above instructions for “Transferring all relevant files”.

ssh "${TARGET_USER}@${TARGET_IP}"
        Copy to clipboard

Note

You will have to login with your target device’s login for that username.
2. Navigate to the directory containing the test files:

cd /data/local/tmp/inception_v3
        Copy to clipboard
3. Run the following command on the target device to execute an inference:

./qnn-net-run \
           --model "./libInception_v3.so" \
           --input_list "./target_raw_list.txt" \
           --backend "./libQnnCpu.so" \
           --output_dir "./output"
        Copy to clipboard
4. Run the following script on the target device to view the classification results:

Note

You can alternatively copy the output folder back to your host machine with `scp` and run the following script there to avoid having to install python on your target device.

python ".\show_inceptionv3_classifications.py" `
            -i ".\cropped\raw_list.txt" `
            -o "output" `
            -l ".\imagenet_slim_labels.txt"
        Copy to clipboard
5. Verify that the classification results in `output` match the following:
1. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/trash_bin.raw 0.777344 413 ashcan`
2. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/chairs.raw 0.253906 832 studio couch`
3. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/plastic_cup.raw 0.980469 648 measuring cup`
4. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/notice_sign.raw 0.167969 459 brass`

### GPU

#### Transferring over all relevant files

1. On the target device, open a terminal and make a destination folder by running:

mount -o remount,rw /
        mkdir -p /data/local/tmp
        cd /data/local/tmp
        ln -s /etc/ /data/local/tmp
        chmod -R 777 /data/local/tmp
        mkdir -p /data/local/tmp/inception_v3
        Copy to clipboard
2. On the host device, use `scp` to transfer `libQnnGpu.so` from your host machine to `/data/local/tmp/inception_v3` on the target device.

scp "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnGpu.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
3. Use `scp` to transfer the example built model.
- Update the `x64` folder below to the proper folder for your built model. The folder name depends on your host machine’s architecture.

scp "/tmp/qnn_tmp/model_libs/x64/libInception_v3.so"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
4. Transfer the input data, input list, and script from the QNN SDK examples folder into `/data/local/tmp/inception_v3` on the target device using `scp` in a similar way:

scp -r "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/target_raw_list.txt"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/imagenet_slim_labels"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/scripts/show_inceptionv3_classifications.py"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
5. Transfer `qnn-net-run` from `$QNN_SDK_ROOT/bin/$QNN_TARGET_ARCH/qnn-net-run` to `/data/local/tmp/inception_v3` on the target device:

scp "$QNN_SDK_ROOT/bin/$QNN_TARGET_ARCH/qnn-net-run" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard

#### Doing inferences on the target device processor

1. Open a PowerShell instance on the target Windows device.
- Alternatively, you can `ssh` from your Linux host machine, run the following command to `ssh` into your target device.
- These console variables were set in the above instructions for “Transferring all relevant files”.

ssh "${TARGET_USER}@${TARGET_IP}"
        Copy to clipboard

Note

You will have to login with your target device’s login for that username.
2. Navigate to the directory containing the test files:

cd /data/local/tmp/inception_v3
        Copy to clipboard
3. Run the following command on the target device to execute an inference:

./qnn-net-run \
           --model "./libInception_v3.so" \
           --input_list "./target_raw_list.txt" \
           --backend "./libQnnGpu.so" \
           --output_dir "./output"
        Copy to clipboard
4. Run the following script on the target device to view the classification results:

Note

You can alternatively copy the output folder back to your host machine with `scp` and run the following script there to avoid having to install python on your target device.

python ".\show_inceptionv3_classifications.py" `
            -i ".\cropped\raw_list.txt" `
            -o "output" `
            -l ".\imagenet_slim_labels.txt"
        Copy to clipboard
5. Verify that the classification results in `output` match the following:
1. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/trash_bin.raw 0.777344 413 ashcan`
2. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/chairs.raw 0.253906 832 studio couch`
3. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/plastic_cup.raw 0.980469 648 measuring cup`
4. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/notice_sign.raw 0.167969 459 brass`

### DSP

Warning

DSP processors require quantized models instead of full precision models. If you do not have a quantized model, please follow here of the CNN to QNN tutorial to build one.

#### Transferring over all relevant files

1. On the target device, open a terminal and make a destination folder by running:

mount -o remount,rw /
        mkdir -p /data/local/tmp
        cd /data/local/tmp
        ln -s /etc/ /data/local/tmp
        chmod -R 777 /data/local/tmp
        mkdir -p /data/local/tmp/inception_v3
        Copy to clipboard
2. Determine your target device’s SnapDragon architecture by looking your chipset up in the [Supported Snapdragon Devices](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/QNN_general_overview.html#supported-snapdragon-devices) table.
3. Update the “X” values below and run the commands to set `DSP_ARCH` to match the version number found in the above table.
1. Only the 2 digits at the end should update, and they should have the same version. Ex. For “V68”, the proper value would be `hexagon-v68`.

$DSP_VERSION="XX"
        $DSP_ARCH="hexagon-v${DSP_VERSION}"
        Copy to clipboard
4. Use `scp` to transfer `libQnnDsp.so` as well as other necessary executables from your host machine to `/data/local/tmp/inception_v3` on the target device.

scp "$QNN_SDK_ROOT/lib/${QNN_TARGET_ARCH}/libQnnDsp.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/lib/${DSP_ARCH}/unsigned/libQnnDspV${DSP_VERSION}Skel.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnDspV${DSP_VERSION}Stub.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/model_libs/${QNN_TARGET_ARCH}/*" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
5. Check the [Backend table](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/backend.html) to see if there are any other processor-specific executables needed for your target processor (`DSP`) and your target device’s architecture (`$QNN_TARGET_ARCH`).
1. Use similar syntax above for `scp` to transfer any additional `.so` files listed **below** your selected target architecture in [this table](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/backend.html). **(There may be none!)**

Warning

Ensure you `scp` the `hexagon-v##` values (in addition to the other architecture files!)
6. Use `scp` to transfer the example built model.
1. Update the `x64` folder below to the proper folder for your built model. The folder name depends on your host machine’s architecture.

scp "/tmp/qnn_tmp/model_libs/x64/libInception_v3.so"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
7. Transfer the input data, input list, and script from the QNN SDK examples folder into `/data/local/tmp/inception_v3` on the target device using `scp` in a similar way:

scp -r "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/target_raw_list.txt"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/imagenet_slim_labels"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/scripts/show_inceptionv3_classifications.py"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
8. Transfer `qnn-net-run` from `$QNN_SDK_ROOT/bin/$QNN_TARGET_ARCH/qnn-net-run` to `/data/local/tmp/inception_v3` on the target device:

scp "${QNN_SDK_ROOT}/bin/${QNN_TARGET_ARCH}/qnn-net-run" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard

#### Doing inferences on the target device processor

1. Open a PowerShell instance on the target device.
1. Alternatively, you can `ssh` from your host machine, run the following command to `ssh` into your target device.
2. These console variables were set in the above instructions for “Transferring all relevant files”.

ssh "${TARGET_USER}@${TARGET_IP}"
        Copy to clipboard

Note

You will have to log in with your target device’s login for that username.
2. Navigate to the directory containing the test files:

cd /data/local/tmp/inception_v3
        Copy to clipboard
3. Run the following command on the target device to execute an inference:

./qnn-net-run \
           --model "./libInception_v3.so" \
           --input_list "./target_raw_list.txt" \
           --backend "./libQnnDsp.so" \
           --output_dir "./output"
        Copy to clipboard
4. Run the following script on the target device to view the classification results:

Note

You can alternatively copy the output folder back to your host machine with `scp` and run the following script there to avoid having to install python on your target device.

python3 ".\show_inceptionv3_classifications.py" \
            -i ".\cropped\raw_list.txt" \
            -o "output" \
            -l ".\imagenet_slim_labels.txt"
        Copy to clipboard
5. Verify that the classification results in `output` match the following:
1. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/trash_bin.raw 0.777344 413 ashcan`
2. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/chairs.raw 0.253906 832 studio couch`
3. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/plastic_cup.raw 0.980469 648 measuring cup`
4. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/notice_sign.raw 0.167969 459 brass`

### HTP

Warning

HTP processors require quantized models instead of full precision models. If you do not have a quantized model, please follow here of the CNN to QNN tutorial to build one.

#### Additional HTP Required Setup

*Running the model on a target device’s HTP requires the generation of a \*\*serialized context\**.\*

**On the Host Machine:**

1. Navigate to the directory where you built the model in the previous steps:

cd /tmp/qnn_tmp
        Copy to clipboard
2. Users can set the custom options and different performance modes to HTP Backend through the backend config. Please refer to [QNN HTP Backend Extensions](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/htp_backend.html) for various options available in the config.
3. Refer to the example below for creating a backend config file for the QCS6490/QCM6490 target with mandatory options passed in:
1. Update the following information based on your device’s `htp_arch`.

{
            "graphs": [
                {
                    "graph_names": [
                        "Inception_v3"
                    ],
                    "vtcm_mb": 2
                }
            ],
            "devices": [
                {
                    "htp_arch": "v68"
                }
            ]
        }
        Copy to clipboard
4. The above config file with minimum parameters such as backend extensions config specified through JSON is given below:

{
            "backend_extensions": {
                "shared_library_path": "path_to_shared_library",  // give path to shared extensions library (.dll)
                "config_file_path": "path_to_config_file"         // give path to backend config
            }
        }
        Copy to clipboard
5. To generate the context, update `<path to JSON of backend extensions>` below with the config you wrote above, then run the command:

& "$QNN_SDK_ROOT/bin/${QNN_TARGET_ARCH}/qnn-context-binary-generator" `
            --backend "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/QnnHtp.dll" `
            --model "${QNN_SDK_ROOT}/examples/Models/InceptionV3/model_libs/${QNN_TARGET_ARCH}/libInception_v3.so" `
            --binary_file "libInception_v3.serialized" `
            --config_file <path to JSON of backend extensions>
        Copy to clipboard
6. This creates the serialized context at:
- `${QNN_SDK_ROOT}/examples/Models/InceptionV3/output/libInception_v3.serialized.bin`

#### Transferring over all relevant files

1. On the target device, open a terminal and make a destination folder by running:

mount -o remount,rw /
        mkdir -p /data/local/tmp
        cd /data/local/tmp
        ln -s /etc/ /data/local/tmp
        chmod -R 777 /data/local/tmp
        mkdir -p /data/local/tmp/inception_v3
        Copy to clipboard
2. Determine your target device’s SnapDragon architecture by looking your chipset up in the [Supported Snapdragon Devices](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/QNN_general_overview.html#supported-snapdragon-devices) table.
3. Update the “X” values below and run the commands to set `HTP_ARCH` to match the version number found in the above table.
1. Only the 2 digits at the end should update, and they should have the same version. Ex. For “V68”, the proper value would be `hexagon-v68`.

$HTP_VERSION="XX"
        $HTP_ARCH="hexagon-v${HTP_VERSION}"
        Copy to clipboard
4. Use `scp` to transfer `libQnnHtp.so` as well as other necessary executables from your host machine to `/data/local/tmp/inception_v3` on the target device.

scp "$QNN_SDK_ROOT/lib/${QNN_TARGET_ARCH}/libQnnHtp.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/lib/${DSP_ARCH}/unsigned/*" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnHtpV${HTP_VERSION}Stub.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/output/Inception_v3.serialized.bin" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
5. Check the [Backend table](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/backend.html) to see if there are any other processor-specific executables needed for your target processor (`DSP`) and your target device’s architecture (`$QNN_TARGET_ARCH`).
1. Use similar syntax above for `scp` to transfer any additional `.so` files listed **below** your selected target architecture in [this table](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/backend.html). **(There may be none!)**
2. If you would like to build the model on the HTP device, ensure you also send `scp "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnHtpPrepare.so" "/data/local/tmp/inception_v3"`

Warning

Ensure you transfer the `hexagon-v##` values (in addition to the other architecture files!)
6. Use `scp` to transfer the example built model.
1. Update the `x64` folder below to the proper folder for your built model. The folder name depends on your host machine’s architecture.

scp "/tmp/qnn_tmp/model_libs/x64/libInception_V3.so"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
7. Transfer the input data, input list, and script from the QNN SDK examples folder into `/data/local/tmp/inception_v3` on the target device using `scp` in a similar way:

scp -r "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/target_raw_list.txt"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/imagenet_slim_labels"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/scripts/show_inceptionv3_classifications.py"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
8. Transfer `qnn-net-run` from `$QNN_SDK_ROOT/bin/$QNN_TARGET_ARCH/qnn-net-run` to `/data/local/tmp/inception_v3` on the target device:

scp "${QNN_SDK_ROOT}/bin/${QNN_TARGET_ARCH}/qnn-net-run" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/inception_v3"
        Copy to clipboard
9. On the target device, set the environment variables:

export LD_LIBRARY_PATH="/data/local/tmp/inception_v3"
        export ADSP_LIBRARY_PATH="/data/local/tmp/inception_v3"
        Copy to clipboard

#### Doing inferences on the target device processor

1. Open a terminal instance on the target device.
1. Alternatively, you can `ssh` from your host machine by doing the below command.
2. These console variables were set in the above instructions for “Transferring all relevant files”.

ssh "${TARGET_USER}@${TARGET_IP}"
        Copy to clipboard

Note

You will have to log in with your target device’s login for that username.
2. On your target device, navigate to the directory containing the test files:

cd /data/local/tmp/inception_v3
        Copy to clipboard
3. Run the following command on the target device to execute an inference:

./qnn-net-run \
           --backend "libQnnHtp.so" \
           --input_list "target_raw_list.txt" \
           --retrieve_context "Inception_v3.serialized.bin"
           --output_dir "./output"
        Copy to clipboard
4. Run the following script on the target device to view the classification results:

Note

You can alternatively copy the output folder back to your host machine with `scp` and run the following script there to avoid having to install python on your target device.

python3 ".\show_inceptionv3_classifications.py" \
            -i ".\cropped\raw_list.txt" \
            -o "output" \
            -l ".\imagenet_slim_labels.txt"
        Copy to clipboard
5. Verify that the classification results in `output` match the following:
1. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/trash_bin.raw 0.777344 413 ashcan`
2. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/chairs.raw 0.253906 832 studio couch`
3. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/plastic_cup.raw 0.980469 648 measuring cup`
4. `${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/notice_sign.raw 0.167969 459 brass`

### LPAI

Warning

LPAI Backend on the x86 Windows platform can be used for offline model preparation and hardware simulation.
The execution of serialized model is supported by QNN SDK directly upon Linux Target platform only.

#### Preparing LPAI Configuration Files for Model Preparation

EXAMPLE of `config.json` file:

{
       "backend_extensions": {
          "shared_library_path": "${QNN_SDK_ROOT}/lib/x86_64-windows-msvc/QnnLpaiNetRunExtensions.dll",
          "config_file_path": "./lpaiParams.conf"
       }
    }
    Copy to clipboard

EXAMPLE of `lpaiParams.conf` file that includes only preparation parameters:

{
       "lpai_backend": {
          "target_env": "adsp",
          "enable_hw_ver": "v5"
       },
       "lpai_graph": {
          "prepare": {
             "enable_batchnorm_fold": true,
             "exclude_io": false
          }
       }
    }
    Copy to clipboard

To configure `lpaiParams.conf`, consider using the following optional settings:

lpai_backend
       "target_env"              "arm/adsp/x86/tensilica, default adsp"
       "enable_hw_ver"           "v4,v5 default v5"
    lpai_graph
       prepare
          "enable_batchnorm_fold"   "true/false,     default false"
          "exclude_io"              "true/false,     default false"
    Copy to clipboard

Using the above `config.json` and `lpaiParams.conf` you can use `qnn-context-binary-generator` to build the LPAI offline model.

When files are mentioned, ensure that they have the relative or absolute path to that value.

cd ${QNN_SDK_ROOT}/examples/QNN/converter/models
    $LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${QNN_SDK_ROOT}/lib/x86_64-linux-clang \
    & ${QNN_SDK_ROOT}/bin/x86_64-windows-msvc/qnn-context-binary-generator.exe \
                  --backend ${QNN_SDK_ROOT}/lib/x86_64-windows-msvc/QnnLpai.dll \
                  --model ${QNN_SDK_ROOT}/examples/Models/InceptionV3/model_libs/x86_64-windows-msvc/<QnnModel.dll> \
                  --config_file <config.json> \
                  --log_level verbose \
                  --binary_file <lpai_graph_serialized>
    Copy to clipboard

Note

- Use generated **lpai\_graph\_serialized.bin** file in QNN format to be executed directly by QNN SDK on Linux target.

#### Running LPAI Emulation Backend on Windows x86

While LPAI Backend on x86\_64 Windows CPU is designed for offline model generation as described above,
it still can run in HW simulation mode where internally it executes prepare and execution steps together.

EXAMPLE of `config.json` file for ARM:

{
       "backend_extensions": {
          "shared_library_path": "${QNN_SDK_ROOT}/lib/x86_64-windows-msvc/QnnLpaiNetRunExtensions.dll",
          "config_file_path": "./lpaiParams.conf"
       }
    }
    Copy to clipboard

EXAMPLE of `lpaiParams.conf` file that includes preparation and execution parameters:

{
       "lpai_backend": {
          "target_env": "x86",
          "enable_hw_ver": "v5"
       },
       "lpai_graph": {
          "prepare": {
             "enable_batchnorm_fold": false,
             "exclude_io": false
          },
          "execute": {
             "fps": 1,
             "ftrt_ratio": 10,
             "client_type": "real_time",
             "affinity": "soft",
             "core_selection": 0
          }
       }
    }
    Copy to clipboard

To configure `lpaiParams.conf`, consider using the following optional settings:

lpai_backend
       "target_env"              "arm/adsp/x86/tensilica, default adsp"
       "enable_hw_ver"           "v4,v5 default v5"
    lpai_graph
       prepare
          "enable_batchnorm_fold"   "true/false,     default false"
          "exclude_io"              "true/false,     default false"
       execute
          "fps"                     "Specify the fps rate number, used for clock voting, default 1"
          "ftrt_ratio"              "Specify the ftrt_ratio number, default 10"
          "client_type"             "real_time/non_real_time, defult real_time"
          "affinity"                "soft/hard, default soft"
          "core_selection"          "Specify the core number, default 0"
    Copy to clipboard

Using the above `config.json` and `lpaiParams.conf` you can use `qnn-net-run` to directly execute LPAI backend in simulation mode.

With the appropriate libraries compiled, `qnn-net-run` is used with the following:

Note

If full paths are not given to `qnn-net-run`, all libraries must be added to
LD\_LIBRARY\_PATH and be discoverable by the system library loader.

$ cd ${QNN_SDK_ROOT}/examples/QNN/converter/models
    $ ${QNN_SDK_ROOT}/bin/x86_64-windows-msvc/qnn-net-run.exe \
                  --backend ${QNN_SDK_ROOT}/lib/x86_64-windows-msvc/QnnLpai.dll \
                  --model ${QNN_SDK_ROOT}/examples/QNN/example_libs/x86_64-windows-msvc/libqnn_model_8bit_quantized.dll \
                  --input_list ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt \
                  --config_file <config.json>
    Copy to clipboard

Outputs from the run will be located at the default ./output directory.

#### Running LPAI on Linux Target Platform via ARM Backend using offline prepared graph

While graph prepare is predefined by offline generation step the Execution paramaters still can be changed at the execution time.

EXAMPLE of `config.json` file:

{
       "backend_extensions": {
          "shared_library_path": "./libQnnLpaiNetRunExtensions.so",
          "config_file_path": "./lpaiParams.conf"
       }
    }
    Copy to clipboard

EXAMPLE of `lpaiParams.conf` file that includes only execution parameters:

{
       "lpai_backend": {
          "target_env": "adsp",
          "enable_hw_ver": "v5"
       },
       "lpai_graph": {
          "execute": {
             "fps": 1,
             "ftrt_ratio": 10,
             "client_type": "real_time",
             "affinity": "soft",
             "core_selection": 0
          }
       }
    }
    Copy to clipboard

To configure `lpaiParams.conf`, consider using the following optional settings:

lpai_graph
       execute
          "fps"                     "Specify the fps rate number, used for clock voting, default 1"
          "ftrt_ratio"              "Specify the ftrt_ratio number, default 10"
          "client_type"             "real_time/non_real_time, defult real_time"
          "affinity"                "soft/hard, default soft"
          "core_selection"          "Specify the core number, default 0"
    Copy to clipboard

Using the above `config.json` and `lpaiParams.conf` you can use by `qnn-net-run` to directly execute LPAI backend on target.

Note

Running the LPAI Backend on an Android via ARM target is supported only for **offline prepared graphs**.
Follow **“Preparing LPAI Configuration Files Model Preparation”** paragraph first to prepare the graph on x86\_64 host and then push the serialized context binary to the device.

In order to run on a particular target platform, the libraries compiled for that target must be used.
Below are examples. The QNN\_TARGET\_ARCH variable can be used to specify the appropriate library for the target.

$ export QNN_TARGET_ARCH=aarch64-android
    $ export DSP_ARCH=hexagon-v79
    $ export DSP_VER=V79
    $ export HW_VER=v5
    Copy to clipboard

**To execute the LPAI backend on Android device the following conditions must be met:**

1. ${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned/libQnnLpaiSkel.so has to be signed by client
    2. qnn-net-run to be executed with root permissions
    Copy to clipboard

After offline model preparation of `./output/lpai_graph_serialized.bin` push the serialized model to device:
.. code-block:

$ adb push ./output/lpai_graph_serialized.bin /data/local/tmp/LPAI
    Copy to clipboard

Push configuration files to device:

$ adb push ./config.json /data/local/tmp/LPAI
    $ adb push ./lpaiParams.conf /data/local/tmp/LPAI
    Copy to clipboard

Push LPAI artifacts to device:

$ adb push ${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnLpai.so /data/local/tmp/LPAI
    $ adb push ${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnLpaiNetRunExtensions.so /data/local/tmp/LPAI
    $ adb push ${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnLpaiStub.so /data/local/tmp/LPAI
    $ # Additionally, the LPAI requires Hexagon specific libraries
    $ adb push ${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned/libQnnLpaiSkel.so /data/local/tmp/LPAI
    Copy to clipboard

Push the input data and input lists to device:

$ adb push ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_data_float /data/local/tmp/LPAI
    $ adb push ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt /data/local/tmp/LPAI
    Copy to clipboard

Push the `qnn-net-run` tool:

$ adb push ${QNN_SDK_ROOT}/bin/aarch64-android/qnn-net-run /data/local/tmp/LPAI
    Copy to clipboard

Setup the environment on device:

$ adb shell
    $ cd /data/local/tmp/LPAI
    $ export LD_LIBRARY_PATH=/data/local/tmp/LPAI
    $ export ADSP_LIBRARY_PATH="/data/local/tmp/LPAI"
    Copy to clipboard

Finally, use `qnn-net-run` for execution with the following:

$ ./qnn-net-run --backend libQnnLpai.so --retrieve_context lpai_graph_serialized.bin --input_list input_list_float.txt --config_file <config.json>
    Copy to clipboard

#### Running LPAI on Linux Target Platform via Native aDSP Backend using offline prepared graph

While graph prepare is predefined by offline generation step the Execution paramaters still can be changed at the execution time.

EXAMPLE of `config.json` file:

{
       "backend_extensions": {
          "shared_library_path": "./libQnnLpaiNetRunExtensions.so",
          "config_file_path": "./lpaiParams.conf"
       }
    }
    Copy to clipboard

EXAMPLE of `lpaiParams.conf` file that includes only execution parameters:

{
       "lpai_backend": {
          "target_env": "adsp",
          "enable_hw_ver": "v5"
       },
       "lpai_graph": {
          "execute": {
             "fps": 1,
             "ftrt_ratio": 10,
             "client_type": "real_time",
             "affinity": "soft",
             "core_selection": 0
          }
       }
    }
    Copy to clipboard

To configure `lpaiParams.conf`, consider using the following optional settings:

lpai_graph
       execute
          "fps"                     "Specify the fps rate number, used for clock voting, default 1"
          "ftrt_ratio"              "Specify the ftrt_ratio number, default 10"
          "client_type"             "real_time/non_real_time, defult real_time"
          "affinity"                "soft/hard, default soft"
          "core_selection"          "Specify the core number, default 0"
    Copy to clipboard

Using the above `config.json` and `lpaiParams.conf` you can use by `qnn-net-run` to directly execute LPAI backend on target.

Note

Running the LPAI Backend on an Android via native aDSP target is supported only for **offline prepared graphs**.
Follow **“Preparing LPAI Configuration Files Model Preparation”** paragraph first to prepare the graph on x86\_64 host and then push the serialized context binary to the device.

In order to run on a particular target platform, the libraries compiled for that target must be used.
Below are examples. The QNN\_TARGET\_ARCH variable can be used to specify the appropriate library for the target.

$ export QNN_TARGET_ARCH=aarch64-android
    $ export DSP_ARCH=hexagon-v79
    $ export DSP_VER=V79
    $ export HW_VER=v5
    Copy to clipboard

**To execute the LPAI backend on Android device the following conditions must be met:**

1. The following artifacts in ${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned  has to be signed by client
       a. libQnnLpai.so
       b. libQnnLpaiNetRunExtensions.so
    1. The following artifacts in ${QNN_SDK_ROOT}/lib/${DSP_ARCH}/unsigned  has to be signed by client
       a. libQnnHexagonSkel_dspApp.so
       b. libQnnNetRunDirect${DSP_VER}Skel.so
    2. qnn-net-run to be executed with root permissions
    Copy to clipboard

After offline model preparation of `./output/lpai_graph_serialized.bin` push the serialized model to device:
.. code-block:

$ adb push ./output/lpai_graph_serialized.bin /data/local/tmp/LPAI
    Copy to clipboard

Push configuration files to device:

$ adb push ./config.json /data/local/tmp/LPAI
    $ adb push ./lpaiParams.conf /data/local/tmp/LPAI
    Copy to clipboard

Push the necessary libraries to device:

$ adb push ${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned/libQnnLpai.so /data/local/tmp/LPAI
    $ adb push ${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned/libQnnLpaiNetRunExtensions.so /data/local/tmp/LPAI
    $ adb push ${QNN_SDK_ROOT}/lib/${DSP_ARCH}/unsigned/libQnnHexagonSkel_dspApp.so /data/local/tmp/LPAI
    $ adb push ${QNN_SDK_ROOT}/lib/${DSP_ARCH}/unsigned/libQnnNetRunDirect${DSP_VER}Skel.so /data/local/tmp/LPAI
    Copy to clipboard

Push the input data and input lists to device:

$ adb push ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_data_float /data/local/tmp/LPAI
    $ adb push ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt /data/local/tmp/LPAI
    Copy to clipboard

Push the `qnn-net-run` tool and its dependent libraries:

$ adb push ${QNN_SDK_ROOT}/bin/${QNN_TARGET_ARCH}/qnn-net-run /data/local/tmp/LPAI
    $ adb push ${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnNetRunDirect${DSP_VER}Stub.so /data/local/tmp/LPAI
    Copy to clipboard

Setup the environment on device:

$ adb shell
    $ cd /data/local/tmp/LPAI
    $ export LD_LIBRARY_PATH=/data/local/tmp/LPAI
    $ export ADSP_LIBRARY_PATH="/data/local/tmp/LPAI"
    $ export DSP_VER=V79
    Copy to clipboard

Finally, use `qnn-net-run` for execution with the following:

$ ./qnn-net-run --backend libQnnLpai.so --direct_mode adsp --retrieve_context qnn_model_8bit_quantized.serialized.bin --input_list input_list_float.txt --config_file <config.json>
    Copy to clipboard

Last Published: Jun 04, 2026

Previous Topic
 
CNN to QNN for Windows Host Next Topic

CNN to QNN for Windows Host on Windows Target