# How to Use the QAIRT Sample App In this tutorial we will walk you from start to finish getting a model working with the QAIRT Sample App. The Sample App provides a reference implementation for how to use the QAIRT API to execute inferences on your target device with your model. After you understand how to build the app and use it, you can use its implementation as a starting point for allowing your own on-target application to interact with your model. The recommended developer flow is: 1. Follow these steps to prove you can: 1. Execute your model on the target device. 2. Build and use an app on your target device. 2. Adapt sections of the Sample App into your on-device application for your use case (or add your own logic to expand the Sample App based on your needs). 1. You should read the code in the Sample App to understand what is going on. You can use the API docs to identify how you can change the code for your use case. This tutorial as a whole is designed to show you how to use QAIRT across different host devices, target devices, and situations. In order to provide actionable commands for all of those paths, this tutorial is structured like a “choose your own adventure” where you will frequently be able to choose the steps that are relevant for your situation. Note Throughout this tutorial, we frequently use commands and environment variables to help you take the right actions. If any command does not work for you, we recommend you copy it and any errors into an AI chat of your choosing and ask the AI to explain what the command is doing and why it was not working for you. Before you start taking actions, it’s important for you to note down the configuration(s) you want to support. Having those written down ahead of time will help you choose the right paths through this tutorial for your situation. You can also use the default configuration (underneath these questions) if you aren’t sure yet what configuration you want. ## Questions to Answer Warning If you don’t know what answers to give, you can skip past these questions and use the default configuration underneath. 1. What AI model do you want to use? 1. Qualcomm’s software allows you to work with your own models, or use models from various popular frameworks. You can also explore [Qualcomm’s AI Hub](https://aihub.qualcomm.com/) to find models for your use case. 2. Example model: [EfficientNet Lite](https://github.com/onnx/models/tree/main/validated/vision/classification/efficientnet-lite4/model) 2. Which AI framework is the model using? 1. You can usually tell what framework a model was built with by looking at the file extension of it. 2. Ex. [ONNX](https://onnx.ai/) (`.onnx`), [Tensorflow](https://www.tensorflow.org/) (`.pb`, `.tflite` for TensforflowLite format), [PyTorch](https://pytorch.org/) (`.pt`) etc. 3. What is the architecture and OS of your host machine? 1. You can run the following commands to learn more about your device: 1. On Linux: `uname -a` for architecture and `cat /etc/os-release` for the OS. 2. On Windows: `systeminfo` for architecture and `Get-ComputerInfo | Select-Object WindowsProductName, WindowsVersion, OsArchitecture, OsName` for the OS. 2. Ex. `x86_64` `Ubuntu 22.04` 4. What is the architecture and OS of your target device? 1. See the previous question for commands you can run to investigate this. 2. Ex. `ARM 64` `Android` ## Default Configuration You can use this to experience the QAIRT workflow for running inferences with a model on a target device: 1. Model: [EfficientNet Lite](https://github.com/onnx/models/tree/main/validated/vision/classification/efficientnet-lite4/model) 2. Framework: ONNX 3. Host machine architecture and OS: Use your current dev computer (either Linux or Windows, there is no support for Mac as a host machine) 4. Target device: The same as your host device (testing locally) Note When you are presented with choices going forward, refer back to your answers to these questions. ## Final Notes Before Starting Warning As of now, the steps in this guide are written primarily for a Linux host machine. These are still the right sequence of steps for Windows. To use this with a Windows machine, you should: 1. Use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) to be able to use linux-style commands. Alternatively, you can use an AI chat along with the below rules to adapt commands to Powershell equivalents - the overall flow will be equivalent. 2. Follow the [Windows Setup](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/windows_setup.html) steps before any of these steps. 3. Use `.dll` equivalents of any `.so` file. Note: The filename will be different, as the Linux equivalents normally start with `lib`, while the Windows ones do not. 4. Use `qairt-sample-app.exe` instead of the `qairt-sample-app`. Note Keep in mind that all steps below are **required** unless they explicitly say they are “(optional)”. ## Part 0: Configuring Your System This section is required to ensure you have the proper permissions, software, and environment variables to work with the QAIRT SDK and API. Note Step 3 below defines an aliased function `setenvvar` that we will use extensively throughout this tutorial to set, log, and persist the values across terminal sessions, so ensure you follow those steps. - **Step 1:** (Optional, but recommended) Create a user with proper permissions This step helps you create a non-privileged user with access to `sudo`. While you don’t *need* a custom user, it is recommended to avoid giving installation scripts privileged access. The other advantage to creating this user is it makes a clean workspace you can access anytime with only files relevant to your QAIRT development. We will call the user we create `qairt`. - **Step 1.1:** Create `qairt` user, adding it to the `sudo` group at the same time sudo useradd -mUs $SHELL -G sudo qairt Copy to clipboard - **Step 1.2:** Set the `qairt` user’s password sudo passwd qairt Copy to clipboard - **Step 1.3:** Login to the `qairt` user You’ll need to run this command every time you want to login to the `qairt` user for QAIRT development. su -l qairt Copy to clipboard - **Step 2:** Download and extract the [Qualcomm AI Runtime (QAIRT) SDK](https://www.qualcomm.com/developer/software/neural-processing-sdk-for-ai) In order to work with QAIRT, you’ll need the SDK. The steps below detail how to obtain and extract it. - **Step 2.1:** Visit the [Qualcomm AI Runtime (QAIRT) SDK](https://www.qualcomm.com/developer/software/neural-processing-sdk-for-ai) website. - **Step 2.2:** Right-click the “Get software” button, and click “Copy link” (or a similar button). - **Step 2.3:** Enter `wget ` into the terminal, replacing `` with the URL you just copied. wget https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.34.0.250424/v2.34.0.250424.zip Copy to clipboard - **Step 2.4:** Extract the zip file by running: unzip v*.*.*.*.zip Copy to clipboard - **Step 3:** Setup your QAIRT development environment The steps below explain how to setup your system and terminal environment to work with the QAIRT tools. - **Step 3.1:** Enter the newly created `qairt`’s `bin` directory. cd qairt/*/bin Copy to clipboard - **Step 3.2:** Create an alias for creating environment variables In this guide we’ll be using environment variables often. We’re going to create the alias `setenvvar` to facilitate setting and saving them. New environment variables will be saved/loaded from the new `~/.qairt_vars` file. echo "alias setenvvar='function _setenvvar(){ VAR_NAME=\"\$1\"; VAR_VALUE=\"\$2\"; FILE=\"\$HOME/.qairt_vars\"; touch \"\$FILE\"; grep -q \"^export \$VAR_NAME=\" \"\$FILE\" && sed -i \"s|^export \$VAR_NAME=.*|export \$VAR_NAME=\\\"\$VAR_VALUE\\\"|\" \"\$FILE\" || echo \"export \$VAR_NAME=\\\"\$VAR_VALUE\\\"\" >> \"\$FILE\"; export \$VAR_NAME=\"\$VAR_VALUE\"; echo \"\$VAR_NAME set to \$VAR_VALUE\"; }; _setenvvar'" >> ~/.bashrc && echo "source \$HOME/.qairt_vars" >> ~/.bashrc Copy to clipboard - **Step 3.3:** Grab the appropriate “target” for your target architecture / platform | Target | Architecture | OS/Platform | Toolchain | | --- | --- | --- | --- | | `x86_64-linux-clang` | x86\_64 (64-bit) | Linux | Clang | | `aarch64-android` | ARM64 | Android | Android NDK toolchain | | `aarch64-oe-linux-gcc11.2` | ARM64 | OpenEmbedded Linux | GCC 11.2 | | `aarch64-oe-linux-gcc9.3` | ARM64 | OpenEmbedded Linux | GCC 9.3 | | `aarch64-ubuntu-gcc9.4` | ARM64 | Ubuntu Linux | GCC 9.4 | - **Step 3.4:** Save the target to an environment variable Ensure you’re using the value you obtained from the above step here. setenvvar QAIRT_TARGET_ARCH_AND_OS "x86_64-linux-clang" Copy to clipboard - **Step 3.5:** Add `envsetup.sh` to your local `.bashrc` file and reload it. echo "source $(pwd)/envsetup.sh" >> ~/.bashrc source ~/.bashrc Copy to clipboard You should see something like: [INFO] QAIRT_SDK_ROOT=/home/qairt/qairt/2.47.0.260520 [WARN] QNN_SDK_ROOT/SNPE_ROOT set to QAIRT_SDK_ROOT for backwards compatibility and will be deprecated in a future release. [INFO] QAIRT SDK environment setup complete Copy to clipboard - **Step 3.6:** Install the remaining dependencies via `check-linux-dependency.sh`. Warning This script will prompt you press enter 2-3 times to confirm additional downloads. sudo ${QAIRT_SDK_ROOT}/bin/check-linux-dependency.sh Copy to clipboard - **Step 3.7:** Install the toolchain(s) for the target(s) you plan to build the Sample App for. Pick the sub-step(s) below that match your chosen `QAIRT_TARGET_ARCH_AND_OS`. You only need to set up the toolchains for targets you actually plan to build. - **Sub-Step 3.7.1:** (For `x86_64-linux-clang`) Install Clang 14 The Sample App’s x86\_64 Makefile invokes `clang++-14` directly, so the Clang 14 toolchain must be available on your `PATH`. sudo apt install clang-14 Copy to clipboard Verify the install: clang++-14 --version Copy to clipboard - **Sub-Step 3.7.2:** (For `aarch64-android`) Install the Android NDK and set `ANDROID_NDK_ROOT` The Android target builds via `ndk-build` and requires the Android NDK. The `make all_android` step will fail with `ERROR: ANDROID_NDK_ROOT not set` if this variable is not exported. # Download the Android NDK wget https://dl.google.com/android/repository/android-ndk-r26c-linux.zip # Unzip the NDK unzip android-ndk-r26c-linux.zip # Save ANDROID_NDK_ROOT and add it to PATH setenvvar ANDROID_NDK_ROOT "$(pwd)/android-ndk-r26c" echo 'export PATH="${ANDROID_NDK_ROOT}:${PATH}"' >> ~/.bashrc source ~/.bashrc # Verify your environment variables ${QAIRT_SDK_ROOT}/bin/envcheck -n Copy to clipboard - **Sub-Step 3.7.3:** (For `aarch64-oe-linux-gcc9.3` or `aarch64-oe-linux-gcc11.2`) Install an ARM cross-compiler and set the matching `QAIRT_AARCH64_LINUX_OE_GCC_*` variable The OE Linux Makefiles check for the variables below and abort with `ERROR: QAIRT_AARCH64_LINUX_OE_GCC_ not set` if they are missing. 1. Based on your target device’s GCC version pick one of the following environment variables to set: Warning If none of these apply, you can likely skip this sub-step. This is specifically to set a cross compiler for specific target devices to help build the sample-app executable. | OS | Architecture | GCC Version | Variable Name | | --- | --- | --- | --- | | OE Linux | ARM64 | 11.2 | `QAIRT_AARCH64_LINUX_OE_GCC_112` | | OE Linux | ARM64 | 9.3 | `QAIRT_AARCH64_LINUX_OE_GCC_93` | 2. Download the corresponding cross-compiler from ARM (or use a Qualcomm provided one if you have received one). 1. Go to this page: [https://developer.arm.com/downloads/-/gnu-a](https://developer.arm.com/downloads/-/gnu-a) 2. Search for the GCC version from the above table (ex. Ctrl + F, then search for “9.3”) 3. Find the latest download with that version number. 4. Scan the download section for the proper compiler for your host machine and target device. 1. There will be multiple sections for each type of host machine and each supported OS for your target device. 2. If you cannot find a cross-compiler for your target device, you may want to ask for help in the [Qualcomm Developer Discord](https://discord.com/invite/qualcommdevelopernetwork) to identify the proper cross-compiler for the host machine and target device you are working with. 3. Set the environment variable for your cross-compiler by running the following command with your chosen variable name: setenvvar YOUR_ENV_VARIABLE_NAME_FROM_ABOVE "/path/to/installed/cross/compiler" Copy to clipboard ## Part 1: Build a Model In order to execute an inference using your model on your target device, you need to convert it into a QAIRT Deep Learning Container (`.dlc`) file. The Sample App loads `.dlc` files via the `--input_dlc` flag. Choose which type of model / conversion you would like to do, your options are: 1. Build with an example ONNX Model (Recommended to see the conversion steps) 2. Build your own model or use a pre-built model (ex. Tensorflow, PyTorch) Skim to the **Option** you chose below and follow all steps within: - **Option 1:** Build with an example ONNX Model - **Step 1:** Enter the models directory cd ${QAIRT_SDK_ROOT}/examples/Models Copy to clipboard - **Step 2:** Install numpy, onnx, onnxruntime, onnxsim, and pandas. pip3 install numpy onnx onnxruntime onnxsim pandas Copy to clipboard - **Step 3:** Obtain an ONNX Model You can use whichever model you want, but as an example this guide uses [EfficientNet Lite](https://github.com/onnx/models/tree/main/validated/vision/classification/efficientnet-lite4/model). You’ll likely want a packaged model (like `.tar.gz` or `.zip`) to have access to both the model files and sample input data. - **Step 3.1:** Grab the download link for [EfficientNet Lite](https://github.com/onnx/models/tree/main/validated/vision/classification/efficientnet-lite4/model) 1. Navigate to [EfficientNet Lite](https://github.com/onnx/models/tree/main/validated/vision/classification/efficientnet-lite4/model) in your web browser. 2. Left-click `efficientnet-lite4-11.tar.gz`. 3. Right-click “Raw” in the top-right and click “Copy link address”. - **Step 3.2:** Download the model using wget. wget https://github.com/onnx/models/raw/refs/heads/main/validated/vision/classification/efficientnet-lite4/model/efficientnet-lite4-11.tar.gz Copy to clipboard - **Step 3.3:** Extract the model package. tar -xf *.tar.gz Copy to clipboard - **Step 4:** Save model path to an environment variable In this step we want to save the model path to an environment variable for future use. This is the file that ends in `.onnx`. setenvvar ONNX_MODEL_PATH "${QAIRT_SDK_ROOT}/examples/Models/efficientnet-lite4/efficientnet-lite4.onnx" Copy to clipboard - **Step 5:** Get model dimensions and name - **Step 5.1:** Retrieve model dimensions and name Run this script to get the input name and dimensions python3 -c "import os, onnx, onnxruntime; \ f = os.environ['ONNX_MODEL_PATH']; \ m = onnx.load(f); \ s = onnxruntime.InferenceSession(f); \ lines = [f'ONNX Input: name={i.name}, shape={[d.dim_value for d in i.type.tensor_type.shape.dim]}\n' for i in m.graph.input] print(''.join(lines), end=''); \ open('input_name_and_dim.txt', 'w').writelines(lines)" Copy to clipboard You can access these values later by looking at `input_name_and_dim.txt` - **Step 5.2:** Save model dimensions and name to environment variables eval $(sed -n 's/ONNX Input: name=\([^,]*\), shape=\[\(.*\)\]/setenvvar ONNX_INPUT_NAME "\1"; setenvvar ONNX_INPUT_DIMENSIONS "\2"/p' ${ONNX_MODEL_PATH%/*}/input_name_and_dim.txt) Copy to clipboard You should see the following output: ONNX_INPUT_NAME set to images:0 ONNX_INPUT_DIMENSIONS set to 1, 224, 224, 3 Copy to clipboard - **Step 6:** Create input\_list.txt For running the model we need input data. The QAIRT tools expect this data to be in a raw format, and the paths defined in a text file. - **Step 6.1:** Convert inputs to raw If your input data is in Protobuf format (`*.pb`) it’s going to need to be converted. Note This script assumes data in a path of `${ONNX_MODEL_PATH%/*}/test_data_set_*/input_0.pb`, edit it to suit your path if needed. python3 -c ' import onnx, numpy as np, struct, glob, os onnx_model_path = os.environ["ONNX_MODEL_PATH"] base_dir = os.path.dirname(onnx_model_path) pattern = os.path.join(base_dir, "test_data_set_*/input_0.pb") for pb in glob.glob(pattern): tensor = onnx.TensorProto() with open(pb, "rb") as f: tensor.ParseFromString(f.read()) arr = onnx.numpy_helper.to_array(tensor).astype(np.float32) raw_path = os.path.splitext(pb)[0] + ".raw" arr.tofile(raw_path) print("Wrote", raw_path, arr.shape, arr.nbytes, "bytes") ' Copy to clipboard You should see the following output: Wrote /home/qairt/qairt/2.47.0.260520/examples/Models/efficientnet-lite4/test_data_set_0/input_0.raw (1, 224, 224, 3) 602112 bytes Wrote /home/qairt/qairt/2.47.0.260520/examples/Models/efficientnet-lite4/test_data_set_2/input_0.raw (1, 224, 224, 3) 602112 bytes Wrote /home/qairt/qairt/2.47.0.260520/examples/Models/efficientnet-lite4/test_data_set_1/input_0.raw (1, 224, 224, 3) 602112 bytes Copy to clipboard - **Step 6.2:** Create a file containing every input path ls -la "${ONNX_MODEL_PATH%/*}" | grep '^d' | awk '{print $9}' | grep -vE '^\.\.?$' | awk -v dir="${ONNX_MODEL_PATH%/*}" '{print dir "/" $0 "/input_0.raw"}' > "${ONNX_MODEL_PATH%/*}/input_list.txt" Copy to clipboard - **Step 6.3:** Save `input_list.txt` to an environment variable setenvvar QAIRT_INPUT_LIST "${ONNX_MODEL_PATH%/*}/input_list.txt" Copy to clipboard - **Step 7:** Convert the model to a DLC with `qairt-converter` ${QAIRT_SDK_ROOT}/bin/x86_64-linux-clang/qairt-converter \ --input_network "${ONNX_MODEL_PATH}" \ -d "${ONNX_INPUT_NAME}" "${ONNX_INPUT_DIMENSIONS}" \ -l "${ONNX_INPUT_NAME}" NHWC \ --output_path "${ONNX_MODEL_PATH%.*}.dlc" Copy to clipboard - **Step 8:** Save model path setenvvar QAIRT_MODEL_PATH "${ONNX_MODEL_PATH%.*}.dlc" Copy to clipboard You should see something like the following output: QAIRT_MODEL_PATH set to /home/qairt/qairt/2.47.0.260520/examples/Models/efficientnet-lite4/efficientnet-lite4.dlc Copy to clipboard - **Option 2:** Build your own model or use a pre-built model (ex. Tensorflow, PyTorch) - **Step 1:** Enter the models directory cd ${QAIRT_SDK_ROOT}/examples/Models Copy to clipboard - **Step 2:** Install dependencies - **Step 2.1:** Install numpy and pandas (required by most model frameworks). pip3 install numpy pandas Copy to clipboard - **Step 2.2:** Choose which framework files are relevant to you: | Package | Version | Description | | --- | --- | --- | | [PyTorch (torch)](https://pypi.org/project/torch/) | 1.13.1 | PyTorch is used for building and training deep learning models with a focus on flexibility and speed. Used with .pt files. **Install by downloading the proper binary** [here](https://pytorch.org/get-started/previous-versions/#v1130) **if pip install does not work.** | | [Torchvision (torchvision)](https://pypi.org/project/torchvision/) | 0.14.1 | Torchvision is used for computer vision tasks with PyTorch, providing datasets, model architectures, and image transforms. | | [TensorFlow (tensorflow)](https://pypi.org/project/tensorflow/) | 2.10.1 | Tensorflow is used for building and training machine learning models, particularly deep learning models. Used with .pb files. NOTE: The `envcheck` script will incorrectly say this file is not installed on Ubuntu. | | [TFLite (tflite)](https://pypi.org/project/tflite/) | 2.3.0 | TFLite is used for running TensorFlow models on mobile and edge devices with optimized performance. Used with .tflite files. | | [ONNX (onnx)](https://pypi.org/project/onnx/) | 1.12.0 | ONNX stands for Open Neural Network Exchange. It is used for defining and exchanging deep learning models between different frameworks. Used with .onnx files. | | [ONNX Runtime (onnxruntime)](https://pypi.org/project/onnxruntime/) | 1.17.1 | ONNX stands for Open Neural Network Exchange. It is used for running ONNX models with high performance across various hardware platforms. Used with .onnx files. | | [ONNX Simplifier (onnxsim)](https://pypi.org/project/onnxsim/) | 0.4.36 | Onnxsim is used for simplifying ONNX models to reduce complexity and improve inference efficiency. Used with .onnx files. | - **Step 2.3:** Install the framework specific packages from above, for example: pip3 install tensorflow Copy to clipboard - **Step 3:** Obtain Your Model You will need to install both your model file and input data to test with. You can [find models here](https://aihub.qualcomm.com/ecosystem) or bring your own. You’ll likely want a packaged model (like `.tar.gz` or `.zip`) to have access to sample input data. - **Step 3.1:** Download the model, for example: wget https://github.com/onnx/models/raw/refs/heads/main/validated/vision/classification/efficientnet-lite4/model/efficientnet-lite4-11.tar.gz Copy to clipboard - **Step 3.2:** Extract the model. - **Step 3.3:** Download any test data you will want to use. - **Step 4:** Save model path to an environment variable In this step we want to save the model path to an environment variable for future use. This is the file containing your model (e.g. `.onnx`, `.pb`, `.tflite`, `.pt`). setenvvar MODEL_PATH "${QAIRT_SDK_ROOT}/examples/Models/efficientnet-lite4/efficientnet-lite4.onnx" Copy to clipboard - **Step 5:** Get model dimensions and name Later we will need the model dimensions (ex. `1,299,299,1` for our Tensorflow example) and the name of the layer we want to output during inference (ex. `Reshape_1`). The way to identify these values varies from framework to framework. So, we recommend you use AI to generate a script to help inspect your model files (or hand-code an interpreter using the model framework package). - **Step 5.1:** Generate code to inspect your model’s dimensions and layer names. - **Step 5.1.1:** Update the “MODEL FRAMEWORK” value and copy this into an AI chat (like ChatGPT) to generate a script you can use to inspect your model: MODEL FRAMEWORK: TARGET OS TO RUN THIS IN: Linux (Ubuntu) Please generate a python script to inspect the model dimensions and layers for my model file that is using the MODEL FRAMEWORK specified above. Use the same environment variables as in the example below - they will be set to the proper values for my model. Also use the same output file. Here is an example of how this script should be implemented for ONNX: python3 -c "import os, onnx, onnxruntime; \ f = os.environ['MODEL_PATH']; \ m = onnx.load(f); \ s = onnxruntime.InferenceSession(f); \ lines = [f'Input: name={i.name}, shape={[d.dim_value for d in i.type.tensor_type.shape.dim]}\n' for i in m.graph.input] print(''.join(lines), end=''); \ open('input_name_and_dim.txt', 'w').writelines(lines)" Copy to clipboard - **Step 5.1.2:** Run your script to inspect your model file. You should be able to find the name and shape later by looking at `input_name_and_dim.txt`. - **Step 5.2:** Save model dimensions and name to environment variables eval $(sed -n 's/Input: name=\([^,]*\), shape=\[\(.*\)\]/setenvvar INPUT_NAME "\1"; setenvvar INPUT_DIMENSIONS "\2"/p' ${MODEL_PATH%/*}/input_name_and_dim.txt) Copy to clipboard You should see the following output: INPUT_NAME set to images:0 INPUT_DIMENSIONS set to 1, 224, 224, 3 Copy to clipboard - **Step 6:** Create `input_list.txt` For running the model we need input data. The QAIRT tools expect this data to be in a raw format, and the paths defined in a text file. - **Step 6.1:** Convert inputs to .raw (Optional - may be needed if your inputs are not .raw files) This is only required if your input files are protobuf files (`.pb`). - **Step 6.1.1:** Update the “MODEL FRAMEWORK” value and copy this into an AI chat (like ChatGPT) to generate a script you can use to convert your input data: MODEL FRAMEWORK: TEST_DATA_FORMAT: test_data_set_*/input_0.pb TARGET OS TO RUN THIS IN: Linux (Ubuntu) Please generate a python script to convert input files for my model located at TEST_DATA_FORMAT into the .raw format.. Use the same environment variables as in the example below - they will be set to the proper values for my model. Also use the same output file. Here is an example of how this script should be implemented for ONNX: python3 -c ' import onnx, numpy as np, struct, glob, os model_path = os.environ["MODEL_PATH"] base_dir = os.path.dirname(model_path) pattern = os.path.join(base_dir, "test_data_set_*/input_0.pb") for pb in glob.glob(pattern): tensor = onnx.TensorProto() with open(pb, "rb") as f: tensor.ParseFromString(f.read()) arr = onnx.numpy_helper.to_array(tensor).astype(np.float32) raw_path = os.path.splitext(pb)[0] + ".raw" arr.tofile(raw_path) print("Wrote", raw_path, arr.shape, arr.nbytes, "bytes") ' Copy to clipboard - **Step 6.2:** Create an `input_list.txt` file containing every input path The following command assumes all directories are input directories containing “input\_0.raw”, outputs list to `input_list.txt`: ls -la "${MODEL_PATH%/*}" | grep '^d' | awk '{print $9}' | grep -vE '^\.\.?$' | awk -v dir="${MODEL_PATH%/*}" '{print dir "/" $0 "/input_0.raw"}' > "${MODEL_PATH%/*}/input_list.txt" Copy to clipboard - **Step 6.3:** Save `input_list.txt` to an environment variable setenvvar QAIRT_INPUT_LIST "${MODEL_PATH%/*}/input_list.txt" Copy to clipboard - **Step 7:** Convert the model to a DLC with `qairt-converter` `qairt-converter` auto-detects the framework from the file extension, so the same tool handles ONNX, TensorFlow, TensorFlow Lite, and PyTorch inputs. ${QAIRT_SDK_ROOT}/bin/x86_64-linux-clang/qairt-converter \ --input_network "${MODEL_PATH}" \ -d "${INPUT_NAME}" "${INPUT_DIMENSIONS}" \ -l "${INPUT_NAME}" NHWC \ --output_path "${MODEL_PATH%.*}.dlc" Copy to clipboard - **Step 8:** Save model path setenvvar QAIRT_MODEL_PATH "${MODEL_PATH%.*}.dlc" Copy to clipboard You should see something like the following output: QAIRT_MODEL_PATH set to /home/qairt/qairt/2.47.0.260520/examples/Models/efficientnet-lite4/efficientnet-lite4.dlc Copy to clipboard ## Part 2: Build the QAIRT Sample App Now that our environment’s setup and we have a converted model, we’re ready to build the QAIRT Sample App. The app source ships with the SDK under `${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp`. - **Step 1:** Enter the Sample App example directory cd ${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp Copy to clipboard - **Step 2:** Build the app by following the instructions for one of the below target device architectures: - **Option 1:** Linux (x86\_64) make all_x86 Copy to clipboard This will produce an executable which can be checked via the `file` command: file "${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp/bin/x86_64-linux-clang/qairt-sample-app" Copy to clipboard You should see something like: /home/qairt/qairt/2.47.0.260520/examples/QAIRT/SampleApp/bin/x86_64-linux-clang/qairt-sample-app: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, not stripped Copy to clipboard - **Option 2:** Android (aarch64) make all_android Copy to clipboard This will produce an executable which can be checked via the `file` command: file "${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp/bin/aarch64-android/qairt-sample-app" Copy to clipboard You should see something like: /home/qairt/qairt/2.47.0.260520/examples/QAIRT/SampleApp/bin/aarch64-android/qairt-sample-app: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, stripped Copy to clipboard ## Part 3: Moving the app and binaries to the target device In this part we’re going to use `rsync` over SSH to transfer files between your devices. Your target device must be running an SSH server, and you’ll need the following: - Target device IP address - Target device username - Target device password - **Step 1:** Enter input\_list directory cd ${QAIRT_INPUT_LIST%/*} Copy to clipboard - **Step 2:** Set environment variables based on your target device’s OS: - **Option 1:** Linux (SSH) In this step you’ll set your target machine’s IP and username for transferring later, as well as the destination path. The target user will need SSH access. Ensure you’re replacing the `*_GOES_HERE` snippets with your info. setenvvar QAIRT_TARGET_ADDR TARGET_IP_GOES_HERE setenvvar QAIRT_TARGET_USER TARGET_USER_GOES_HERE setenvvar QAIRT_TARGET_DEST /tmp/qairt Copy to clipboard - **Option 2:** Android (adb) For Android we only need to set the target for our sample app and related binaries. setenvvar QAIRT_TARGET_DEST /data/local/tmp/qairt Copy to clipboard - **Step 3:** Create new input\_list file We’re moving our files to a new location, this new input\_list will contain our new paths. Note This assumes that the input data is in the same directory as your input list. If you have data in sub-folders, you will need to modify this. awk -F'/' -v dest="$QAIRT_TARGET_DEST" '{print dest "/" $(NF-1) "/" $NF}' input_list.txt > input_list_target.txt Copy to clipboard - **Step 4**: Create environment variable file for target We need to create an environment variable file to locate our files on the new machine. echo "export QAIRT_INPUT_LIST=${QAIRT_TARGET_DEST}/input_list_target.txt" > ./target_env_vars.env echo "export QAIRT_SAMPLE_APP=${QAIRT_TARGET_DEST}/qairt-sample-app" >> ./target_env_vars.env echo "export QAIRT_MODEL_PATH=${QAIRT_TARGET_DEST}/${QAIRT_MODEL_PATH##*/}" >> ./target_env_vars.env Copy to clipboard - **Step 5:** Move files to target device Note The following section assumes your input data is within the same directory as your `input_list.txt`. Pick the proper OS based on your target device: - **Option 1:** Linux rsync -av $(awk -F'/' '{print $(NF-1)}' input_list_target.txt) input_list_target.txt target_env_vars.env ${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp/bin/${QAIRT_TARGET_ARCH_AND_OS}/qairt-sample-app ${QAIRT_SDK_ROOT}/lib/${QAIRT_TARGET_ARCH_AND_OS}/libQairtCpu.so ${QAIRT_MODEL_PATH} ${QAIRT_TARGET_USER}@${QAIRT_TARGET_ADDR}:${QAIRT_TARGET_DEST} Copy to clipboard - **Option 2:** Android Create and run this script (`android_adb.sh`) to copy over all the files needed via adb for the sample app: DEST=/data/local/tmp/qairt # Create the target dir adb shell "mkdir -p $DEST" # Push dirs listed in input_list_target.txt for dir in $(awk -F'/' '{print $(NF-1)}' input_list_target.txt | sort -u); do adb push "$dir" "$DEST/" done # Push input_list_target.txt and env file adb push input_list_target.txt "$DEST/" adb push target_env_vars.env "$DEST/" # Push Sample App binary and libQairtCpu.so adb push "${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp/bin/${QAIRT_TARGET_ARCH_AND_OS}/qairt-sample-app" "$DEST/" adb push "${QAIRT_SDK_ROOT}/lib/${QAIRT_TARGET_ARCH_AND_OS}/libQairtCpu.so" "$DEST/" # Push model adb push "${QAIRT_MODEL_PATH}" "$DEST/" Copy to clipboard Then run it: ./android_adb.sh Copy to clipboard ## Part 4: Run the QAIRT Sample App At this stage we’ll run the app and generate an output. Below are 2 options: **Host** and **Target**. If you want to test your changes on your host machine, follow the “Host Machine” steps. If you want to run the sample app on your target device, follow the “On your Target Device” steps. The Sample App requires three arguments: `--backend` (path to a backend `.so`), `--input_dlc` (path to your converted DLC), and `--input_list` (path to your input-list file). - **Option 1:** On your Host Machine (for testing locally) - **Step 1:** Enter the Sample App directory. cd ${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp/ Copy to clipboard - **Step 2:** Run the Sample App. - **Sub-Option 1:** Linux (x86\_64) ${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp/bin/x86_64-linux-clang/qairt-sample-app \ --backend ${QAIRT_SDK_ROOT}/lib/x86_64-linux-clang/libQairtCpu.so \ --input_dlc ${QAIRT_MODEL_PATH} \ --input_list ${QAIRT_INPUT_LIST} Copy to clipboard - **Sub-Option 2:** Android (aarch64) ${QAIRT_SDK_ROOT}/examples/QAIRT/SampleApp/bin/aarch64-android/qairt-sample-app \ --backend ${QAIRT_SDK_ROOT}/lib/aarch64-android/libQairtCpu.so \ --input_dlc ${QAIRT_MODEL_PATH} \ --input_list ${QAIRT_INPUT_LIST} Copy to clipboard - **Option 2:** On your Target Device - **Step 1:** Install `libc++1` This step assumes you’re running Ubuntu 22.04. If not, install the equivalent C++ runtime for your distribution. apt install libc++1 Copy to clipboard - **Step 2:** Enter temporary qairt directory In **Part 3** we transferred all our required files into `$QAIRT_TARGET_DEST`, this destination is different on Linux or Android systems. Choose the option based on your target device’s OS: - **Sub-Option 1:** Linux cd /tmp/qairt Copy to clipboard - **Sub-Option 2:** Android cd /data/local/tmp/qairt Copy to clipboard - **Step 3:** Load the environment variables source target_env_vars.env Copy to clipboard - **Step 4:** Run the sample app ./qairt-sample-app \ --backend ./libQairtCpu.so \ --input_dlc ${QAIRT_MODEL_PATH} \ --input_list ${QAIRT_INPUT_LIST} Copy to clipboard ## Conclusion With that, you have successfully ran the QAIRT Sample App on your target device! The next steps are to: 1. Inspect the source code for the Sample App in order to understand how it uses the QAIRT API to interact with your model. 2. Integrate similar logic into your on-target application to use your model. 1. You can also build off of the Sample App if you don’t have a pre-existing application. You will likely need to leverage the API section to look up what various functions do. Last Published: Jun 04, 2026 [Previous Topic Overview](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/QAIRT-API_overview.md) [Next Topic Migration Guide](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/migration-guide.md)