# Object detection and display with ONNX The use cases use an ONNX object detection model to identify objects in a scene from a single camera stream. The detected bounding boxes are either overlaid on the video output or composed into the rendered stream and displayed. Run the use case on the target device: export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/dev/socket/weston && \ gst-launch-1.0 -v filesrc location=/etc/media/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12 ! tee name=split \ split. ! queue ! qtivcomposer name=mixer ! queue ! waylandsink fullscreen=true \ split. ! queue ! qtimlvconverter ! queue ! qtimlonnx model=/etc/models/model.onnx execution-provider=qnn backend-path="/usr/lib/libQnnHtp.so" ! queue ! qtimlpostprocess module=yolov8 labels=/etc/labels/yolox.json ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer. Copy to clipboard Note When using an `ONNXModel`, place the model weight file in the same directory as the ONNX model file and name it `model.data`. To stop the use case, use **CTRL + C**. The following figure shows the flow of the use case execution: 1. Identifies object scenes in the scene from a video stream, which is coming through a camera source. 2. Overlays bounding boxes over the detected objects using overlaylib. 3. Displays the results. **Figure : Pipeline for object detection with ONNX** The following table provides the sequential processing stages of the pipeline execution: | Process | Description | | --- | --- | | [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70029-50/topic/qtiqmmfsrc.html) |
Collects the video stream (source) and creates two copies of the source:
One stream is sent to qtimetamux plugin to retain the video stream.
The other stream is sent to an ML inferencing pipeline.
Receives the video stream on its sink pad.
Performs preprocessing:
Color conversion
Scaling down/up
Normalization on the stream data when the model expects the floating point values as input
Converts the video stream to a tensor stream on its source pad.
The object detection model uses this tensor stream for inferencing.
Loads the object detection model.
Modifies the graph for the chosen delegate.
Receives the tensor stream on its sinkpad.
Runs the inference and produces a tensor stream with the object detection results on its source pad.
Receives the inference tensors from object detection.
Converts the inference tensors on its sinkpad into formats like video or text that the multimedia plugins can process later.
Applies the threshold to the chosen number of results.
Loads the corresponding modules for detection models.
In this use case, qtimlpostprocess does the following:
Loads the YOLOv8 submodule.
Produces results as structures of text.
Sends them to the sinkpad of qtimetamux.
Receives video stream and text stream with bounding box results corresponding to the video stream on its sinkpads.
Produces GST buffers with contents of video stream from its sink pad.
Adds bounding boxes as GstVideoRegionOfInterest from data sinkpad to GST buffers meta (meta muxing) on its source pad.
Receives the multiplexed stream.
Overlays the bounding boxes on the VideoFrame using CL.
Produces GST buffers with overlays in its source pad.
Receives the video stream on its sinkpad.
Submits the video stream to Weston.
Weston renders the video stream and bounding boxes generated for the objects in that scene on a local display device.