# Object detection use case

Source: [https://docs.qualcomm.com/doc/80-88500-3/topic/object_detection_use_case.html](https://docs.qualcomm.com/doc/80-88500-3/topic/object_detection_use_case.html)

- The following use cases are described:
    - Single camera stream: 1080p with AI inference.
    - Single file stream: 720p with AI inference.
- **TensorFlow Lite**: Steps to execute the use cases
    - Display (variant 1): Use `qtioverlay`plug-in to apply
                            bounding box overlay.
        - Camera source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
        - File source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
    - Display (variant 2): Use the `qtivcomposer`plug-in to
                            mix the original frame with the bounding box mask.
        - Camera source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer.Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer.Copy to clipboard
        - File source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer.Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer.Copy to clipboard
    - Encode (variant 1): Use the `qtioverlay`plug-in to apply
                            the bounding box overlay.
        - Camera source:

                gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
        - File source:

                gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/ Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/ yolov5m.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
    - Encode (variant 2): Use the `qtivcomposer`plug-in to mix
                            the original frame with the bounding box mask.
        - Camera source:

                gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\ (memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! qtic2venc target- bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320-
                int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! video/x- raw,format=BGRA,width=640,height=360 ! queue ! mixer.
                Copy to clipboard
        - File source:

                gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/ Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! qtic2venc target- bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/yolov5m-320x320- int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5m labels=/data/yolov5m.labels ! video/x- raw,format=BGRA,width=640,height=360 ! queue ! mixer.Copy to clipboard
- **Qualcomm Neural Processing SDK**: Steps to execute the use cases
    - Display (variant 1): Use `qtioverlay`plug-in to apply
                            detection overlay.
        - Camera source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
        - File source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/draw.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/draw.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
    - Display (variant 2): Use the `qtivcomposer`plug-in to
                            mix the original frame with the detection mask.
        - Camera source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
        - File source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/draw.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/draw.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
    - Encode (variant 1): Use the `qtioverlay`plug-in to apply
                            the detection overlay.
        - Camera source:

                gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
        - File source:

                gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/draw.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
    - Encode (variant 2): Use the `qtivcomposer`plug-in to mix
                            the original frame with the detection mask.
        - Camera source:

                gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
        - File source:

                gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/draw.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/yolov5s_relu_finetune_quantized_cle_bc.dlc layers="<Conv_139, Conv_140, Conv_141>" ! queue ! qtimlvdetection threshold=90.0 results=10 module=yolov5s labels=/data/yolov5s.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
- Validation method
    - The application saves .mp4 files to storage when it receives
                                    CTRL+ C and
                            terminates the command-line tool successfully.
    - To verify the AI processing, check the video stream running on the
                            display. There should be bounding boxes visible around objects such as a
                            person.

**Parent Topic:** [Configure AI/ML modules](https://docs.qualcomm.com/doc/80-88500-3/topic/66_Configure_AI_ML_modules.html)

Last Published: Sep 26, 2023

[Previous Topic
Add GST plug-in to support tensors in and out](https://docs.qualcomm.com/bundle/publicresource/80-88500-3/topics/75_Add_GST_plug_in_to_support_tensors_in_and_out___.md) [Next Topic
Image segmentation use case](https://docs.qualcomm.com/bundle/publicresource/80-88500-3/topics/image_segmentation_use_case.md)