# Run a TensorFlow Lite model using the Gstreamer-based Qualcomm® Intelligent Multimedia
SDK (IM SDK)
Source: [https://docs.qualcomm.com/doc/80-70014-54/topic/run-a-tensorflow-lite-model-using-the-gstreamer-based-qim-sdk.html](https://docs.qualcomm.com/doc/80-70014-54/topic/run-a-tensorflow-lite-model-using-the-gstreamer-based-qim-sdk.html)
The RB3 Gen 2 Vision kit ships with precompiled end-to-end TensorFlow Lite sample
applications to run sample TensorFlow Lite models.
The gst-ai-classification sample application uses IM SDK plug-ins to run a TensorFlow
Lite classification model on the RB3 Gen 2 device with hardware acceleration using
TensorFlow Lite delegates.
Figure : Workflow to run a TensorFlow Lite model using IM SDK
The gst-ai-classification sample application does the following:
1. Opens the IMX577 camera present on the RB3 Gen 2 Vision kit with a specific
resolution and fps, for example, 1080p at 30 fps.
2. Preprocesses each camera frame to input to a classification model.
For example,
the gst-ai-classification sample application does the following:
1. Downscales a 1080p frame to a 224 x 224 resolution.
2. Normalizes the input frame based on the model requirements.
3. The qtimltflite IM SDK plug-in, which is written on top of the TensorFlow Lite C++
API, does the following:
1. Loads the sample TensorFlow Lite classification model.
2. Performs inference on the model provided with hardware acceleration.
4. Postprocesses the output from the inference, that is, extract the label with highest
predicted probability within the output tensor.
5. Overlays the inference result on the original camera input image and displays it on
the connected monitor.
## Download sample models
To download sample models and copy them to the RB3 Gen 2 Vision kit, do the
following:
1. To download sample models and the corresponding label files, run the following
command:
wget https://github.com/quic/sample-apps-for-qualcomm-linux/releases/download/v0.1.0/v0.1.0.tar.gzCopy to clipboard
2. To extract files, run the following
command:
tar -zxvf v0.1.0.tar.gzCopy to clipboard
3. To copy the models and label files to the device using the Secure Copy Protocol
(SCP), run the following
commands:
# For SCP, run the following command:
ssh root@[ip-addr]
setenforce 0
mount -o remount,rw /
exitCopy to clipboard
# Copy files securely
scp v0.1.0/* root@[ip-addr]:/opt/Copy to clipboard
Note: When prompted for a password, enter
oelinux123.
## Execute TensorFlow Lite model with sample application
1. To run inference using TensorFlow Lite Runtime, run the following
commands:
ssh root@[ip-addr]
setenforce 0Copy to clipboard
# Copy inceptionv3.tflite to inceptionv3_uint8.tflite
cp /opt/inceptionv3.tflite /opt/inceptionv3_uint8.tfliteCopy to clipboard
# Setup Wayland Display environment
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
# Run a classification sample app
gst-ai-classification --ml-framework=2Copy to clipboard
- Default model path:
/opt/inceptionv3\_uint8.tflite
- Default label path: /opt/classification.labels
2. To run the sample application using a custom classification model, use the
following arguments:
- `--model`
- `--labels`
For
example:
gst-ai-classification --ml-framework=2 --model=/opt/model.tflite --labels=/opt/classification.labelsCopy to clipboard
3. To stop the use case, press CTRL+C.
The sample application, when running, displays the camera stream on the connected
monitor with inference results overlaid on the frame.
**Parent Topic:** [Getting started](https://docs.qualcomm.com/doc/80-70014-54/topic/getting-started.html)
Last Published: Jul 12, 2024
[Previous Topic
Getting started](https://docs.qualcomm.com/bundle/publicresource/80-70014-54/topics/getting-started.md) [Next Topic
Run a TensorFlow Lite model using a native TensorFlow Lite sample application](https://docs.qualcomm.com/bundle/publicresource/80-70014-54/topics/run-a-tensorflow-lite-model-using-a-native-tensorflow-lite-sample-application.md)