# Image classification
Source: [https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-classification.html](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-classification.html)
The **gst-ai-classification** application allows you to identify the subject in an
image. The use cases use the Qualcomm Neural Processing SDK, LiteRT, or Qualcomm AI Engine
direct models.
The figure shows the pipeline, which receives a video stream from a camera, file source,
or Real-Time Streaming Protocol (RTSP), does the preprocessing, runs the inference on
the AI hardware, and displays the results.
For information about the plugins used for classification, see Pipeline flow.
Figure : gst-ai-classification pipeline
## Sample model and label files
Table : Sample model and label files for gst-ai-classification
| Runtime | Model files | Label files |
| :--- | :--- | :--- |
| Qualcomm Neural Processing SDK | inceptionv3.dlc | classification.labels |
| LiteRT | inception_v3_quantized.tflite |
| Qualcomm AI Engine direct | inception_v3_quantized.bin |
## Prerequisites
- If not already done so, [Download and install eSDK](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-51/install-sdk.html#download-and-install-esdk-).
- To run the application, push the model and label files to the device. For
instructions, see Download model and label files.
The
application supports the Qualcomm Neural Processing SDK, Qualcomm AI Engine
direct, and LiteRT models.
- To access your host computer, enable SSH. For instructions, see [Sign in using SSH](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-254/how_to.html#use-ssh).
Note: If SSH is already enabled, you can skip this
step.
- Push the model files from the Linux host computer:
scp root@:/etc/modelsCopy to clipboard
- Note that the downloaded
script downloads the sample video.mp4 video to
the /etc/media directory. If you are using a custom video,
then ensure that you push the video to /etc/media and
update the file path in the application config.JSON file.
- Connect the display to the device using the HDMI port. For instructions, see
[Set up HDMI display](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/samples.html).
- Enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
If you face issues while enabling camera or display, see [Camera troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-17/troubleshooting.html) and [Display troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/debug.html).
## Run the application
The sample application uses the
/etc/configs/config\_classification.json file to read the
input parameters.
To create your own config JSON file, use [config_classification.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-classification/config_classification.json?ref_type=heads) as a
reference.
1. Use the following format of the config\_classification.json
file.
{
"file-path": "",
"ml-framework": "",
"model": "",
"labels": "",
"threshold": ,
"constants": "",
"runtime": ""
}Copy to clipboard
Note: Update the config JSON file based on
the model, input stream, and other properties. For more information, see
Config JSON field description.
For example, run the application using input from a file, LiteRT model, DSP runtime, custom
constants, and custom threshold
value:
{
"file-path": "/etc/media/video.mp4",
"ml-framework": "tflite",
"model": "/etc/models/inception_v3_quantized.tflite",
"labels": "/etc/labels/classification.labels",
"threshold": 40,
"runtime": "dsp",
"constants": "Inceptionv3,q-offsets=<38.0>,q-scales=<0.17039915919303894>;"
}Copy to clipboard
2. Run the gst-ai-classification
application:
gst-ai-classification --config-file=/etc/configs/config_classification.jsonCopy to clipboard
To display the available help options, run the following command in the SSH
shell:
gst-ai-classification -hCopy to clipboard
To stop the use case, use CTRL + C.
## Expected output
The classified object is displayed on the local display.
Figure : Expected output for gst-ai-classification application
## Pipeline flow
The table lists the plugins used in the object classification pipeline:| Plugin | Description |
| --- | --- |
| Camera source:qtiqmmfsrc |
- Captures the live stream from camera.
- Uses tee to split the stream for inferencing.
|
| File source: filesrc |
- Captures the video stream using filesrc, followed by
qtdemux, which demultiplexes the stream.
- Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
- Captures the RTSP stream using rtspsrc, followed by
rtph264depay for video extraction.
- Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| v4l2h264dec | Decodes the video. |
| qtimlvconverter |
- Receives the video stream on its sink pad.
- Performs the following preprocessing on the stream data.
This preprocessing is done when the model expects
floating-point values as input.
- Color conversion
- Scaling (up or down)
- Normalization
- Converts the preprocessed video stream to a tensor stream on
its source pad.
The tensor stream is used for inferencing in the later stages of
the pipeline. |
| Inferencing plugins: |
- After the inference runtime receives the tensor stream on
its sink pad, it runs the inference.
- Produces a tensor stream with the inference results on its
source pad.
|
| qtimlvclassification | Handles inference results from any classification model.
- Applies a threshold to the chosen number of results. For
quantized model, add Softmax and constants (q-offsets and
q-scales).
- Loads the MobileNet postprocessing module.
- Produces results as video frames with classification
labels.
- Sends these processed results to the sink pad of
qtivcomposer.
|
| qtivcomposer |
- Composes frames with contents from its sink pads.
- Pushes the GStreamer buffers containing these composed
frames to its source pad.
|
| Waylandsink |
- Waylandsink submits the video stream received on its sink
pad to Weston.
- Weston renders the video stream on a local display.
|
## Config JSON field description
| Field | Values/description |
| :--- | :--- |
| **ml-framework** | Use one of the following models:
snpe: Qualcomm Neural Processing SDK
tflite: LiteRT
qnn: Qualcomm AI Engine direct
|
| **runtime** | Use one of the following runtimes: |
| **Input source** | Use one of the following input sources:
camera: Primary (0) or secondary (1).
file-path: The directory path to the video
file.
rtsp-ip-port: The address of the RTSP
stream:
rtsp://<ip>:<port>/<stream>
|
## Known issues
The text overlay of the classification object is small.
**Parent Topic:** Run AI/ML sample applications
**Related Resources**
- Image classification and display with LiteRT
- Image classification and encode with LiteRT
- Image classification and display with Neural Processing SDK
- Image classification and encode with Neural Processing SDK
Last Published: Jan 30, 2026
Previous Topic
Download model and label files Next Topic
Object detection