# Object detection
Source: [https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-object-detection.html](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-object-detection.html)
The **gst-ai-object-detection** application allows you to detect objects within
images and videos. The use cases show the execution of [YOLOv5](https://github.com/ultralytics/yolov5), [YOLOv8](https://github.com/ultralytics/ultralytics), and [YOLOX](https://github.com/Megvii-BaseDetection/YOLOX/blob/main/README.md) using the Qualcomm Neural Processing SDK runtime, YOLOv8
using Qualcomm AI Engine direct, and YOLOv5 and YOLOv8 using LiteRT.
The following figure shows the pipeline, which receives the input from a live camera
feed, file, USB source, or an RTSP stream, preprocesses it, runs inferences on AI
hardware. The results are either displayed on the screen, saved as an encoded MP4 file,
or streamed over the RTSP server. For information about the plugins used in the pipeline
flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-object-detection.html#gst-ai-object-detection__section_p2w_33y_kbc).
Figure : gst-ai-object-detection pipeline
Qualcomm
Open source
File (default) Camera (optional) tee
qtdemux h264parse V4l2h264dec qtimlvconverter qtimltflite/ qtimlsnpe/ qtimlqnn qtimlpostprocess sink_1
sink_0
filesrc
qtimlvconverter qtimltflite/ qtimlsnpe/ qtimlqnn qtimlpostprocess sink_1
sink_0
qtiqmmfsrc
RTSP (optional)
sink_0
rtph264 depay h264parse V4l2h264dec tee qtimlvconverter qtimltflite/ qtimlsnpe/ qtimlqnn qtimlpostprocess sink_1 rtspsrc
USB camera (Optional)
tee
qtimlvconverter qtimltflite/ qtimlsnpe/ qtimlqnn qtimlpostprocess sink_1
sink_0
v4l2src_caps v4l2src
qtivcomposer
or or
Waylandsink
qtirtspbin
filesink
## Sample model and label files
| Runtime | Model files | Label files |
| --- | --- | --- |
| Qualcomm Neural Processing SDK | *yolonas.dlc* | yolonas.json yolov8.json yolox.json yolonas.json yolov8.json yolox.json yolonas.json yolov8.json yolox.json ",
"yolo-model-type": "",
"model": "",
"labels": "",
"threshold": "",
"runtime": "",
"output-type": "waylandsink or filesink or rtspsink"
}Copy to clipboard
For example, run the application using input from the video
file, LiteRT, YOLOx model, DSP runtime, and custom threshold
value:
{
"file-path": "/etc/media/video.mp4",
"ml-framework": "tflite",
"yolo-model-type": "yolox",
"model": "/etc/models/yolox_quantized.tflite",
"labels": "/etc/labels/yolox.json",
"threshold": 40,
"runtime": "dsp",
"output-type": "waylandsink"
}Copy to clipboard
4. Run the gst-ai-object-detection application:
gst-ai-object-detection --config-file=/etc/configs/config_detection.jsonCopy to clipboard
Note: For USB camera input, set the
`video-format`, `resolution`, and
`framerate` parameters in the configuration file to
match the capabilities of the camera. To check the camera capabilities,
see [Configure USB camera](https://docs.qualcomm.com/bundle/publicresource/topics/80-70022-8/usb.html#configure-usb-camera).
Note: The
Yolo-NAS-Quantized.tflite model can be run
using the YOLOv8 value in the `yolo-model-type` field.
5. To display the available help options, run the following command in the SSH
shell:
gst-ai-object-detection -hCopy to clipboard
6. To stop the use case, use CTRL +
C.
## Expected output
Figure : Expected output for gst-ai-object-detection application
## Pipeline flow
The following table lists the plugins used in the object detection
pipeline:
| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70022-50/topic/qtiqmmfsrc.html) | Captures the live stream from camera. Uses tee to split the stream for inferencing. Captures the video stream using filesrc, followed by Uses tee to split the stream for inferencing. Captures the RTSP stream using rtspsrc, followed by Uses tee to split the stream for inferencing. Captures the live stream from USB camera. Uses tee to split the stream for inferencing. Receives the video stream on its sink pad. Performs the following preprocessing on the stream data.Color conversion Scaling (up or down) Normalization Converts the preprocessed video stream to a tensor stream on After the inference runtime receives the tensor stream on Produces a tensor stream with the inference results on its Applies a threshold to the chosen number of results. Loads the YOLO (YOLOv5, YOLOv8, or YOLO-NAS) module. Produces video frames with only bounding boxes that can be Sends these processed frames to the sink pad of Composes frames with contents from its sink pads. Pushes the GStreamer buffers containing these composed Waylandsink submits the video stream received on its sink Weston renders the video stream on a local display. Serves as a network sink. Transmits UDP packets to the network. snpe–Qualcomm Neural Processing SDKtflite–LiteRTqnn–Qualcomm AI Engine directcamera–Primary (0) or secondary (1).file-path–The directory path to the videortsp-ip-port–The address of the RTSPrtsp://<ip>:<port>/<stream> enable-usb-camera: Set to TRUE orwaylandsink : To display output onfilesink: To store output in file.rtspsink: To stream output on server.width: Input USB camera sourceheight: Input USB camera sourceframerate: Input USB camera source