# Qualcomm AI Engine Direct Model Preparation on WSL
Source: [https://docs.qualcomm.com/doc/80-64748-1/topic/model_prep_wsl.html](https://docs.qualcomm.com/doc/80-64748-1/topic/model_prep_wsl.html)
qnn_model_prepare_on_wsl
# Qualcomm AI Engine Direct Stable Diffusion Model Preparation on WSL
Download
The Qualcomm AI Engine Direct SDK allows clients to run ML models on HTP hardware. The following steps describe how to prepare and execute the Stable Diffusion models on Windows on Snapdragon (WoS) platforms with HTP capability.
Please note that for the rest of this document the term Qualcomm Neural Network (QNN) will be interchangeably used with Qualcomm AI Engine Direct SDK.
# Prerequisites
1. WSL and Ubunu 20.04 installation and set up with required packages for QNN Tools
2. Qualcomm AI Engine Direct SDK (with Ubuntu Linux and Windows support)
3. Stable diffusion `.onnx` files and their corresponding AIMET encodings (generated via AIMET workflow)
# Tested Environment
**Surface Pro9**
- OS build: 22621.169
- Windows Feature Experience Pack 1000.22632.1000.0
- Windows On Snapdragon : SC8280X
- `qcadsprpc` file version should be 1.0.3530.9800 or greater
**Get qcadsprpc version:**
1. Update Windows OS and Windows driver with the latest version.
2. Go to "System Information" menu and clck "Software Environment->System Drivers". Select `qcadsprpc` on name and check file path.
3. Go to file path on #2.
4. Push left button on `qcadsprpc8280.sys` and go to details.
5. See file version number: it should be 1.0.3530.9800 or above.
# Workflow
This section assumes that you have the following Stable Diffusion model artifacts generated after following the AIMET Stable diffusion workflow:
1. Stable Diffusion text encoder model and its AIMET encodings
2. Stable Diffusion U-Net model and its AIMET encodings
3. Stable Diffusion Variational Auto Encoder (VAE) model and its AIMET encodings
4. `fp32.npy` file - This is a numpy object array saved as a python pickle. It contains data that is required as part of the model conversion step.
The three models and encodings are independently processed via different executable utilities available in the Qualcomm AI Engine Direct SDK.
These QNN (Qualcomm Neural Network) utilities allow the user to prepare the Stable Diffusion models for inference.
The QNN executable utilities for steps 1 through 3 require a WSL Ubuntu 20.04 environment. Step 4 is the inference step and executes in a native Windows environment.
1. Convert the `.onnx` files to their equivalent QNN representation with `A16W8` (16-bit activation and 8-bit weights)
2. Generate the QNN model libraries
3. Generate the QNN context binaries for the QNN HTP backend
4. Execute QNN context binaries for inference on Windows on Snapdragon (WoS) Platform
# Install AI Engine Direct SDK using Qualcomm Package Manager 3
1. Install QPM from [https://www.qualcomm.com/support/support-portals/qualcomm-package-manager](https://www.qualcomm.com/support/support-portals/qualcomm-package-manager) after login.
2. Once installation is complete, select **Tools**.
3. Search for AI Engine Direct SDK in the search bar.
4. Click on the SDK and select the desired version from the dropdown as shown below.
5. Once the SDK installation is complete, you can optionally copy the SDK from its original install location as shown in the final dialog box to the current directory of the notebook.
# WSL and Ubuntu 20.04 Installation
Before installing WSL, enable the Virtual Machine feature in Windows.
**Note**: Execute powershell commands in administrator mode.
In [ ]:
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
Copy to clipboard
Follow these instructions to install WSL: [https://learn.microsoft.com/en-us/windows/wsl/install](https://learn.microsoft.com/en-us/windows/wsl/install)
WSL 2 requires updated kernel components. Download and install the Linux kernel update package from [https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi](https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi)
In [ ]:
# Install WSL and necessary components
wsl --install
# Install Ubuntu 20.04 distribution in WSL
wsl --install -d Ubuntu-20.04
Copy to clipboard
After a successful installation and machine restart, a new terminal window opens with a prompt to set up your Unix username and password for log in and sudo access to Ubuntu. For reference: [https://learn.microsoft.com/en-us/windows/wsl/setup/environment#set-up-your-linux-username-and-password](https://learn.microsoft.com/en-us/windows/wsl/setup/environment#set-up-your-linux-username-and-password)
# WSL Ubuntu 20.04 Setup
Install the required packages to use the QNN Tools in the Ubuntu 20.04 envrionment (Ubuntu Terminal window).
In [ ]:
# Update the package index files
sudo apt-get update
# Install python3.6 and necessary packages
sudo bash -c 'apt-get update && apt-get install software-properties-common && add-apt-repository ppa:deadsnakes/ppa && apt-get install python3.6 python3.6-distutils libpython3.6'
sudo apt-get install python3.6
# Install python3-pip
sudo apt-get install python3-pip
# Install python3 virtual environnment support
sudo apt install python3-virtualenv python3.6-distutils
# Create python3.6 virtual environment
virtualenv -p /usr/bin/python3.6 venv_wsl
source venv_wsl/bin/activate
# Install jupyter notebook to execute cells from notebook interactively
pip install jupyter
# After setting up python and pip in WSL Ubuntu, check QNN tools dependencies.
# Set QNN_SDK_ROOT environment variable to the location of Qualcomm AI Engine Directory for **Linux**
export QNN_SDK_ROOT=./qnn_assets/unzipped_qnn_sdk_linux/
# Check and install Linux dependencies
source $QNN_SDK_ROOT/bin/check-linux-dependency.sh
sudo apt-get install -y libtinfo5
# Check and install Python dependencies
python $QNN_SDK_ROOT/bin/check-python-dependency
# Install ONNX frameworks
pip install "onnx==1.6.0" "onnx-simplifier==0.4.8" "onnxruntime==1.10.0" "packaging"
# Setup necessary environment variables for QNN Tools
source $QNN_SDK_ROOT/bin/envsetup.sh
Copy to clipboard
# Prepare Stable Diffusion Models for Inference
The following section will covers steps 1, 2 and 3 and uses the Qualcomm AI Engine Direct SDK to prepare stable diffusion models for on-target inference.
In [ ]:
# Set up environment variable to reference STABLE_DIFFUSION_MODELS
export STABLE_DIFFUSION_MODELS="./qnn_assets/stable_diffusion_models/"
Copy to clipboard
## Convert the model from ONNX representation to QNN representation
The Qualcomm AI Engine Direct SDK `qnn-onnx-conerter` tool converts a model from ONNX representation to its equivalent QNN representation in `A16W8` precision. The encoding files generated from the AIMET workflow are provided as an input to this step via the `–quantization_overrides model.encodings` option.
This step generates a `.cpp` file that represents the model as a series of QNN API calls and a `.bin` file that contains static data that is typically model weights and referenced by the `.cpp` file.
This step must be done for all three models independently.
### Generate model inputs list/data
In [ ]:
python3 generate_inputs.py --pickle_path --working_dir
Copy to clipboard
### Convert the text encoder
In [ ]:
mkdir $STABLE_DIFFUSION_MODELS/converted_text_encoder
qnn-onnx-converter -o $STABLE_DIFFUSION_MODELS/converted_text_encoder/qnn_model.cpp \
-i $STABLE_DIFFUSION_MODELS/text_encoder_onnx/text_encoder.onnx \
--input_list $STABLE_DIFFUSION_MODELS/text_encoder_onnx/text_encoder_input_list.txt \
--act_bw 16 \
--bias_bw 32 \
--quantization_overrides $STABLE_DIFFUSION_MODELS/text_encoder_onnx/text_encoder.encodings
# Rename the model files to make them unique and helpful for subsequent stages
mv $STABLE_DIFFUSION_MODELS/converted_text_encoder/qnn_model.cpp $STABLE_DIFFUSION_MODELS/converted_text_encoder/text_encoder.cpp
mv $STABLE_DIFFUSION_MODELS/converted_text_encoder/qnn_model.bin $STABLE_DIFFUSION_MODELS/converted_text_encoder/text_encoder.bin
mv $STABLE_DIFFUSION_MODELS/converted_text_encoder/qnn_model_net.json $STABLE_DIFFUSION_MODELS/converted_text_encoder/text_encoder_net.json
Copy to clipboard
### Convert U-Net
Expected execution time: ~35 minutes
In [ ]:
mkdir $STABLE_DIFFUSION_MODELS/converted_unet
qnn-onnx-converter -o $STABLE_DIFFUSION_MODELS/converted_unet/qnn_model.cpp \
-i $STABLE_DIFFUSION_MODELS/unet_onnx/unet.onnx \
--input_list $STABLE_DIFFUSION_MODELS/unet_onnx/unet_input_list.txt \
--act_bw 16 \
--bias_bw 32 \
--quantization_overrides $STABLE_DIFFUSION_MODELS/unet_onnx/unet.encodings \
-l input_3 NONTRIVIAL
# Rename the model files to make them unique and helpful for subsequent stages
mv $STABLE_DIFFUSION_MODELS/converted_unet/qnn_model.cpp $STABLE_DIFFUSION_MODELS/converted_unet/unet.cpp
mv $STABLE_DIFFUSION_MODELS/converted_unet/qnn_model.bin $STABLE_DIFFUSION_MODELS/converted_unet/unet.bin
mv $STABLE_DIFFUSION_MODELS/converted_unet/qnn_model_net.json $STABLE_DIFFUSION_MODELS/converted_unet/unet_net.json
Copy to clipboard
### Convert the variational autoencoder (VAE) decoder
Expected execution time: ~25 minutes
In [ ]:
mkdir $STABLE_DIFFUSION_MODELS/converted_vae_decoder
qnn-onnx-converter -o $STABLE_DIFFUSION_MODELS/converted_vae_decoder/qnn_model.cpp \
-i $STABLE_DIFFUSION_MODELS/vae_decoder_onnx/vae_decoder.onnx \
--input_list $STABLE_DIFFUSION_MODELS/vae_decoder_onnx/vae_decoder_input.txt \
--act_bw 16 \
--bias_bw 32 \
--quantization_overrides $STABLE_DIFFUSION_MODELS/vae_decoder_onnx/vae_decoder.encodings
# Renaming for uniqueness
mv $STABLE_DIFFUSION_MODELS/converted_vae_decoder/qnn_model.cpp $STABLE_DIFFUSION_MODELS/converted_vae_decoder/vae_decoder.cpp
mv $STABLE_DIFFUSION_MODELS/converted_vae_decoder/qnn_model.bin $STABLE_DIFFUSION_MODELS/converted_vae_decoder/vae_decoder.bin
mv $STABLE_DIFFUSION_MODELS/converted_vae_decoder/qnn_model_net.json $STABLE_DIFFUSION_MODELS/converted_vae_decoder/vae_decoder_net.json
Copy to clipboard
## QNN model library
The Qualcomm AI Engine Direct SDK `qnn-model-lib-generator` compiles the model `.cpp` and `.bin` files into a shared object library for a specific target. This example generates a shared object library for x86\_64-linux target.
The inputs to this stage are the `model.cpp` and `model.bin` files that were generated in the previous step.
### Generate the text encoder model library
In [ ]:
qnn-model-lib-generator -c $STABLE_DIFFUSION_MODELS/converted_text_encoder/text_encoder.cpp \
-b $STABLE_DIFFUSION_MODELS/converted_text_encoder/text_encoder.bin \
-t x86_64-linux-clang \
-o $STABLE_DIFFUSION_MODELS/converted_text_encoder
Copy to clipboard
### Generate the U-Net model library
Expected execution time: ~25 minutes
In [ ]:
qnn-model-lib-generator -c $STABLE_DIFFUSION_MODELS/converted_unet/unet.cpp \
-b $STABLE_DIFFUSION_MODELS/converted_unet/unet.bin \
-t x86_64-linux-clang \
-o $STABLE_DIFFUSION_MODELS/converted_unet
Copy to clipboard
### Generate the variational autoencoder (VAE) decoder model library
In [ ]:
qnn-model-lib-generator -c $STABLE_DIFFUSION_MODELS/converted_vae_decoder/vae_decoder.cpp \
-b $STABLE_DIFFUSION_MODELS/converted_vae_decoder/vae_decoder.bin \
-t x86_64-linux-clang \
-o $STABLE_DIFFUSION_MODELS/converted_vae_decoder/
Copy to clipboard
## QNN HTP context binary
The Qualcomm AI Engine Direct SDK `qnn-context-binary-generator` tool creates a QNN context binary applicable to the QNN HTP backend. This binary can be deployed to run on a Windows on Snapdragon platform. This step requires the model shared object library from the previous step and the `libQnnHtp.so` file available in the Qualcomm AI Engine Direct SDK.
Provide any additional options that pertain to the QNN HTP backend by passing the `libQnnHtpBackendExtensions.so` file available in the Qualcomm AI Engine Direct SDK and a configuration file specified in a `.json` format. Documentation on backend extensions and configuraton parameters is available in the Qualcomm AI Engine Direct SDK Docs.
In [ ]:
# Htp backend extensions config file (htp_backend_extensions.json) example
{
"backend_extensions": {
"shared_library_path": "libQnnHtpNetRunExtensions.so",
"config_file_path": "htp_config.json"
}
}
# HTP backend config file (htp_config.json) example for Surface Pro 2
{
"graphs": {
"vtcm_mb":8,
"graph_names":["qnn_model"]
},
"devices": [
{
"soc_id": 43,
"dsp_arch": "v73",
"cores":[{
"core_id": 0,
"perf_profile": "burst",
"rpc_control_latency":100
}]
}
]
}
Copy to clipboard
In [ ]:
# Create a path under models directory for serialized binaries
mkdir $STABLE_DIFFUSION_MODELS/serialized_binaries
Copy to clipboard
### Generate the QNN context binary for text encoder
In [ ]:
qnn-context-binary-generator --model $STABLE_DIFFUSION_MODELS/converted_text_encoder/x86_64-linux-clang/libtext_encoder.so \
--backend libQnnHtp.so \
--output_dir $STABLE_DIFFUSION_MODELS/serialized_binaries \
--binary_file text_encoder.serialized \
--config_file htp_backend_extensions.json
Copy to clipboard
### Generate the QNN context binary for U-Net
Expected execution time: ~2 minutes
In [ ]:
qnn-context-binary-generator --model $STABLE_DIFFUSION_MODELS/converted_unet/x86_64-linux-clang/libunet.so \
--backend libQnnHtp.so \
--output_dir $STABLE_DIFFUSION_MODELS/serialized_binaries \
--binary_file unet.serialized \
--config_file htp_backend_extensions.json
Copy to clipboard
### Generate the QNN context binary for VAE Decoder
Expected execution time: ~1.5 minutes
In [ ]:
qnn-context-binary-generator --model $STABLE_DIFFUSION_MODELS/converted_vae_decoder/x86_64-linux-clang/libvae_decoder.so \
--backend libQnnHtp.so \
--output_dir $STABLE_DIFFUSION_MODELS/serialized_binaries \
--binary_file vae_decoder.serialized \
--config_file htp_backend_extensions.json
Copy to clipboard
Upon completion of these steps to prepare Stable Diffusion models for inference, QNN context binaries for the three models are available in `$STABLE_DIFFUSION_MODELS/serialized_binaries/`
The next step is to execute the prepared models (now represented as serialized context binaries) on a Windows on Snapdragon device using executable utilities available in the Qualcomm AI Engine Direct SDK.
Copyright (c) 2023 Qualcomm Technologies, Inc. and/or its subsidiaries.
**Parent Topic:** [Qualcomm AI Engine Direct Model Preparation](https://docs.qualcomm.com/doc/80-64748-1/topic/model_prep.html)
Last Published: Apr 09, 2024
[Previous Topic
Qualcomm AI Engine Direct Model Preparation](https://docs.qualcomm.com/bundle/publicresource/80-64748-1/topics/model_prep.md) [Next Topic
Qualcomm AI Engine Direct Model Preparation on Linux](https://docs.qualcomm.com/bundle/publicresource/80-64748-1/topics/model_prep_linux.md)