# Object detection
The **gst-ai-object-detection** application allows you to detect objects within images and videos. The use cases show the execution of [YOLOv5](https://github.com/ultralytics/yolov5), [YOLOv8](https://github.com/ultralytics/ultralytics), and [YOLOX](https://github.com/Megvii-BaseDetection/YOLOX/blob/main/README.md) using the Qualcomm Neural Processing SDK runtime, YOLOv8 using Qualcomm AI Engine direct, and YOLOv5 and YOLOv8 using LiteRT.
The following figure shows the pipeline, which receives the input from a live camera feed, file, USB source, or an RTSP stream, preprocesses it, runs inferences on AI hardware. The results are either displayed on the screen, saved as an encoded MP4 file, or streamed over the RTSP server. For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-object-detection.html#section-p2w-33y-kbc).
Qualcomm
Open source
File (default)
Camera (optional)
tee
qtdemux
h264parse
V4l2h264dec
qtimlvconverter
qtimltflite/ qtimlsnpe/ qtimlqnn
qtimlpostprocess
sink_1
sink_0
filesrc
qtimlvconverter
qtimltflite/ qtimlsnpe/ qtimlqnn
qtimlpostprocess
sink_1
sink_0
qtiqmmfsrc
RTSP (optional)
sink_0
rtph264 depay
h264parse
V4l2h264dec
tee
qtimlvconverter
qtimltflite/ qtimlsnpe/ qtimlqnn
qtimlpostprocess
sink_1
rtspsrc
USB camera (Optional)
tee
qtimlvconverter
qtimltflite/ qtimlsnpe/ qtimlqnn
qtimlpostprocess
sink_1
sink_0
v4l2src_caps
v4l2src
qtivcomposer
or
or
Waylandsink
qtirtspbin
filesink
**Figure : gst-ai-object-detection pipeline**
## Input and output capabilities
The following table summarizes the input and output capabilities supported by the sample application:
| Config | Input | Input | Input | Input | Output | Output | Output |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Config | File src | RTSP | USB camera | MIPI camera | File | Display | RTSP |
| Config #1 | Yes | Yes | No | No | No | Yes | Yes |
| Config #2 | Yes | Yes | No | Yes | Yes | Yes | Yes |
| | | | | | | | |
## Sample model and label files
| Runtime | Model files | Label files |
| --- | --- | --- |
| Qualcomm Neural Processing SDK | *yolonas.dlc* |
|
| LiteRT | *yolov8\_det\_quantized.tflite* *yolox\_quantized.tflite* | |
| Qualcomm AI Engine direct | *yolov8\_det\_quantized.bin* | |
| | | |
| | | |
## Run the application on the target device
The sample application uses the `/etc/configs/config_detection.json` file to read the input parameters.
To create your own config JSON file, use [config_detection.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-object-detection/config_detection.json?ref_type=heads) as a reference.
1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-80021-50/topic/download-model-and-label-files.html).
2. Update the config JSON file based on the model, input stream, and other properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-object-detection.html#section-qjx-hqq-32c).
3. Use the following format of the `/etc/configs/config_detection.json` file:
{
"file-path": " ",
"ml-framework": "",
"yolo-model-type": "",
"model": "",
"labels": "",
"threshold": "",
"runtime": "",
"output-type": "waylandsink or filesink or rtspsink"
}
Copy to clipboard
An example format file for Config #2 and Config #1 is as follows:
Tab Config #2
Tab Config #1
For example, run the application using input from the video file, LiteRT, YOLOx model, DSP runtime, and custom threshold value:
{
"file-path": "/etc/media/video.mp4",
"ml-framework": "tflite",
"yolo-model-type": "yolox",
"model": "/etc/models/yolox_quantized.tflite",
"labels": "/etc/labels/yolox.json",
"threshold": 40,
"runtime": "dsp",
"output-type": "waylandsink",
"snpe-tensors": ""
}
Copy to clipboard
For example, run the application using input from the video file, LiteRT, YOLOx model, CPU runtime, and custom threshold value:
{
"file-path": "/etc/media/video.mp4",
"ml-framework": "tflite",
"yolo-model-type": "yolox",
"model": "/etc/models/yolox_quantized.tflite",
"labels": "/etc/labels/yolox.json",
"threshold": 40,
"runtime": "cpu",
"output-type": "waylandsink",
"snpe-tensors": ""
}
Copy to clipboard
Note
Config #1 supports only LiteRT models and the CPU runtime.
Note
The `snpe-tensors` field applies only to the SNPE runtime. To retrieve the output tensor names for a DLC model, open the model in [Netron](https://netron.app/).
Note
When using DLC models from the AI Hub, the `snpe-tensors` field is optional.
4. Run the gst-ai-object-detection application:
gst-ai-object-detection --config-file=/etc/configs/config_detection.json
Copy to clipboard
Note
For USB camera input, set the `video-format`, `resolution`, and `framerate` parameters in the configuration file to match the capabilities of the camera. To check the camera capabilities, see [Configure USB camera](https://docs.qualcomm.com/bundle/publicresource/topics/80-80021-8/usb.html#configure-usb-camera).
Note
The `Yolo-NAS-Quantized.tflite` model can be run using the YOLOv8 value in the `yolo-model-type` field.
5. To display the available help options, run the following command in the SSH shell:
gst-ai-object-detection -h
Copy to clipboard
6. To stop the use case, use **CTRL + C**.
## Expected output

**Figure : Expected output for object detection and display using RTSP stream–Preview**
## Pipeline flow
The following table lists the plugins used in the object detection pipeline:
| Plugin | Description |
| --- | --- |
| Camera source: [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-80021-50/topic/qtiqmmfsrc.html) | |
| File source: filesrc | Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc | Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.
Uses tee to split the stream for inferencing.
|
| USB camera source: v4l2src | |
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlvconverter.html) | Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
Converts the preprocessed video stream to a tensor stream on its source pad.
The tensor stream is used for inferencing in the later stages of the pipeline. |
| Inferencing plugins: | After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
|
| [qtimlpostprocess](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlpostprocess.html) | Handles inference results from any object detection model. 1. Applies a threshold to the chosen number of results. 2. Loads the YOLO (YOLOv5, YOLOv8, or YOLO-NAS) module. 3. Produces video frames with only bounding boxes that can be overlaid on objects. 4. Sends these processed frames to the sink pad of qtivcomposer. |
| [qtivcomposer](https://docs.qualcomm.com/doc/80-80021-50/topic/qtivcomposer.html) | Composes frames with contents from its sink pads.
Pushes the GStreamer buffers containing these composed frames to its source pad.
|
| [Waylandsink](https://docs.qualcomm.com/doc/80-80021-50/topic/waylandsink.html) | Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
| filesink | Receives the video stream on sink pad and saves it as an H.264-encoded MP4 file. |
| qtirtspbin | Serves as a network sink.
Transmits UDP packets to the network.
|
## Config JSON field description
The different parameters available to configure the JSON file and run the use case are as follows:
Table : Field description–config_detection.json file
| Field | Values/description |
| --- | --- |
| **ml-framework** | Use one of the following models: |
| **yolo-model-type** | Run the `yolov5`, `yolov8`, `yolox`, and `yolonas` models respectively. See [Sample model and label files](https://docs.qualcomm.com/doc/80-80021-50/topic/vision_ai_demo_app.html#section-ohr-1qq-32c). |
| **runtime** | Use one of the following runtimes: |
| **Input source** | Use one of the following input sources:camera : Primary (0) or secondary (1).
file-path : The directory path to the video file.
rtsp-ip-port : The address of the RTSP stream: rtsp://<ip>:<port>/<stream>
enable-usb-camera : Set to TRUE or FALSE.
|
| **output-ip-address** | Output server IP address. |
| **port** | Output server port. |
| **output-type** | Use one of the following output-type:waylandsink : To display output on Wayland.
filesink : To store output in file.
rtspsink : To stream output on server.
|
| **snpe-tensors** | `["output-tensor-name","output-tensor-name"]` |
| **USB camera video-format and resolution** | Use one of the following video-format: Use one of the following resolution fields:width : Input USB camera source resolution width.
height : Input USB camera source resolution height.
framerate : Input USB camera source framerate.
|
| **output-file** | Output filename. The default output file is `output_object_detection.mp4`. |
## Known issues
Tab Config #2
Tab Config #1
- Lag is observed in object detection with camera source and file source as quantized models aren't supported in the TFlite IM SDK framework.
- The application may intermittently hang during the `gst_deinit` phase.
- The GPU delegate doesn't function in the QNN and TFLite IM SDK frameworks.
- Bounding box isn't observed with the camera source input.
- USB camera input isn't supported.
- Video encoding use case isn't supported.
## Related information
- [Object detection and display with LiteRT](https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-object-detection-and-display.html)
- [Object detection and encode with LiteRT](https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-object-detection-and-encode.html)
- [Object detection and display with Neural Processing SDK](https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-object-detection-and-display-with-mobilenet-v2-ssd.html)
- [Object detection and encode with Neural Processing SDK](https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-object-detection-and-encode-with-mobilenet-v2-ssd.html)
Last Published: Mar 26, 2026
[Previous Topic
Image classification](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/gst-ai-classification.md) [Next Topic
Pose detection](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/gst-ai-pose-detection.md)
Source: [https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-object-detection.html](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-object-detection.html)