# Image segmentation use case

Source: [https://docs.qualcomm.com/doc/80-88500-3/topic/image_segmentation_use_case.html](https://docs.qualcomm.com/doc/80-88500-3/topic/image_segmentation_use_case.html)

- Use cases
    - Single camera stream: 1080p with AI inference.
    - Single file stream: 720p with AI inference.
- **TensorFlow Lite**: Steps to execute the use cases
    - Display: Use the `qtivcomposer`plug in to mix the
                            original frame with the segmentation mask.
        - Camera source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst- debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM
                    \),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/ dv3_argmax_int32.tflite ! queue ! qtimlvsegmentation module=deeplab- argmax labels=/data/dv3-argmax.labels ! video/x- raw,width=256,height=144 ! queue ! mixer.
                    Copy to clipboard
            - Linux
                                            Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst- debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM
                    \),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/ dv3_argmax_int32.tflite ! queue ! qtimlvsegmentation module=deeplab- argmax labels=/data/dv3-argmax.labels ! video/x- raw,width=256,height=144 ! queue ! mixer.
                    Copy to clipboard
        - File source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst- debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/dv3_argmax_int32.tflite ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/dv3- argmax.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
            - Linux Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst- debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/dv3_argmax_int32.tflite ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/dv3- argmax.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
    - Encode: Use the `qtivcomposer`plug-in to mix the
                            original frame with the segmentation mask.
        - Camera source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/dv3_argmax_int32.tflite ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/dv3-argmax.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
            - Linux Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/dv3_argmax_int32.tflite ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/dv3-argmax.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
        - File source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst- debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/ video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/dv3_argmax_int32.tflite ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/dv3- argmax.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
            - Linux Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst- debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/ video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/dv3_argmax_int32.tflite ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/dv3- argmax.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
- **Qualcomm Neural Processing SDK**: Steps to execute the use cases
    - Display (variant 1): Use the `qtivcomposer`plug-in to
                            mix the original frame with the segmentation mask.
        - Camera source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
            - Linux Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
        - File source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
            - Linux Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
    - Display (variant 2): Use the `qtivcomposer`plug-in to
                            mix the original frame with the detection mask.
        - Camera source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
            - Linux Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
        - File source:
            - Linux
                                            Ubuntu:

                    export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
            - Linux Embedded:

                    export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
    - Encode (variant 1): Use the `qtivcomposer`plug-in to mix
                            the original frame with the detection mask.
        - Camera source:

                gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
        - File source:

                gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=256,height=144 ! queue ! mixer.Copy to clipboard
    - Encode (variant 2): Use the `qtivcomposer`plug-in to mix
                            the original frame with the detection mask.
        - Camera source:

                gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
        - File source:

                gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Draw_720p_180s_24FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::dimensions="<1920,1080>" sink_1::alpha=0.5 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/deeplabv3_quantized.dlc ! queue ! qtimlvsegmentation module=deeplab-argmax labels=/data/deeplabv3.labels ! video/x-raw,width=640,height=360 ! queue ! mixer.Copy to clipboard
- Validation method
    - The application saves .mp4 files to the storage when it receives
                                    CTRL+ C and
                            terminates the command-line tool successfully.
    - To verify the AI processing, check the video stream running on the
                            display. There should be segmentation mask visible over objects such as
                            a person.

**Parent Topic:** [Configure AI/ML modules](https://docs.qualcomm.com/doc/80-88500-3/topic/66_Configure_AI_ML_modules.html)

Last Published: Sep 26, 2023

[Previous Topic
Object detection use case](https://docs.qualcomm.com/bundle/publicresource/80-88500-3/topics/object_detection_use_case.md) [Next Topic
Pose estimation use case](https://docs.qualcomm.com/bundle/publicresource/80-88500-3/topics/pose_estimation_use_case.md)