# Object detection
Source: [https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-object-detection.html](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-object-detection.html)
The **gst-ai-object-detection** application allows you to detect objects within
images and videos. The use cases show the execution of [YOLOv5](https://github.com/ultralytics/yolov5), [YOLOv8](https://github.com/ultralytics/ultralytics), and [YOLOX](https://github.com/Megvii-BaseDetection/YOLOX/blob/main/README.md) using the Qualcomm Neural Processing SDK runtime, YOLOv8
using Qualcomm AI Engine direct, and YOLOv5 and YOLOv8 using LiteRT.
The following figure shows the pipeline, which receives the input from a live camera
feed, file, or an RTSP stream, preprocesses it, runs inferences on AI hardware, and
displays the results on the screen. For information about the plugins used in the
pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-object-detection.html#gst-ai-object-detection__section_p2w_33y_kbc).
Figure : gst-ai-object-detection pipeline
## Sample model and label files
| Runtime | Model files | Label files |
| --- | --- | --- |
| Qualcomm Neural Processing SDK | *yolonas.dlc* | yolonas.labels yolov8.labels yolox.labels yolonas.labels yolov8.labels yolox.labels yolonas.labels yolov8.labels yolox.labels ",
"yolo-model-type": "",
"model": "",
"labels": "",
"constants": "",
"threshold": ,
"runtime": ""
}Copy to clipboard
For example, run the application using input from the video
file, LiteRT, YOLOx model, DSP runtime, custom constants, and custom
threshold value:
{
"file-path": "/etc/media/video.mp4",
"ml-framework": "tflite",
"yolo-model-type": "yolox",
"model": "/etc/models/yolox_quantized.tflite",
"labels": "/etc/labels/yolox.labels",
"constants": "YOLOx,q-offsets=<38.0, 0.0, 0.0>,q-scales=<3.6124823093414307, 0.003626860911026597, 1.0>;",
"threshold": 40,
"runtime": "dsp"
}Copy to clipboard
4. Run the gst-ai-object-detection application:
gst-ai-object-detection --config-file=/etc/configs/config_detection.jsonCopy to clipboard
Note: The
Yolo-NAS-Quantized.tflite model can be run
using the YOLOv8 value in the `yolo-model-type` field.
5. To display the available help options, run the following command in the SSH
shell:
gst-ai-object-detection -hCopy to clipboard
6. To display the available help options, run the following command in the SSH
shell:
gst-ai-object-detection -hCopy to clipboard
7. To stop the use case, use CTRL +
C.
## Expected output
Figure : Expected output for gst-ai-object-detection application
## Pipeline flow
The following table lists the plugins used in the object detection
pipeline:
| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70020-50/topic/qtiqmmfsrc.html) | Captures the live stream from camera. Uses tee to split the stream for inferencing. Captures the video stream using filesrc, followed by Uses tee to split the stream for inferencing. Captures the RTSP stream using rtspsrc, followed by Uses tee to split the stream for inferencing. Receives the video stream on its sink pad. Performs the following preprocessing on the stream data.Color conversion Scaling (up or down) Normalization Converts the preprocessed video stream to a tensor stream on After the inference runtime receives the tensor stream on Produces a tensor stream with the inference results on its Applies a threshold to the chosen number of results. Loads the YOLO (YOLOv5, YOLOv8, or YOLO-NAS) module. Produces video frames with only bounding boxes that can be Sends these processed frames to the sink pad of Composes frames with contents from its sink pads. Pushes the GStreamer buffers containing these composed Waylandsink submits the video stream received on its sink Weston renders the video stream on a local display. snpe–Qualcomm Neural Processing SDKtflite–LiteRTqnn–Qualcomm AI Engine directcamera–Primary (0) or secondary (1).file-path–The directory path to the videortsp-ip-port–The address of the RTSPrtsp://<ip>:<port>/<stream>