# Image segmentation and encode with Neural Processing SDK The use case implements the DeepLab v3 model with the Qualcomm Neural Processing SDK runtime. The use case is to compose the semantic segmentations and original video stream, encode this stream, and then multiplex it in an MP4 container. Run the use case on the target device: gst-launch-1.0 -e --gst-debug=2 \ filesrc location=/opt/Animals_000_1080p_180s_30FPS.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! queue ! tee name=split \ split. ! queue ! qtivcomposer name=mixer sink_1::position="<30, 30>" sink_1::dimensions="<320, 180>" ! queue ! \ v4l2h264enc capture-io-mode=4 output-io-mode=5 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/opt/video.mp4 \ split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe model=/etc/models/deeplabv3_plus_mobilenet-deeplabv3-plus-mobilenet-w8a8.dlc delegate=dsp ! queue ! qtimlpostprocess module=deeplab-argmax labels=/etc/labels/dv3-argmax.json ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer. Copy to clipboard To stop the use case, use **CTRL + C**. The following figure shows the flow of the use case execution: 1. Identify scenes from a video stream coming through a file source. 2. Compose semantic segmentation and video stream using qtivcomposer. 3. Encode the stream as an H.264 bit stream and multiplex the stream in an MP4 container. Qualcomm Open source tee qtimlvconverter qtivcomposer v4l2h264enc h264parse mp4mux filesink qtimlsnpe qtimlpostprocess filesrc qtdemux h264parse v4l2h264dec **Figure : Pipeline for image segmentation and encode with qtivcomposer** The following table provides the sequential processing stages of the pipeline execution: | Process | Description | | --- | --- | | File source: filesrc |

  1. Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.


  2. Uses tee to split the stream for inferencing.


| | h264parse | Parses the H.264 video. | | [v4l2h264dec](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264dec.html) | Decodes the video. | | **Preprocessing** | **Preprocessing** | | [qtimlvconverter](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlvconverter.html) |

  1. Receives the video stream on its sink pad.



  2. Performs preprocessing:

    • Color conversion


    • Scaling down/up


    • Normalization on the stream data when the model expects the floating point values as input






  3. Converts the video stream to a tensor stream on its source pad.


    The segmentation model uses this tensor stream for inferencing.



| | **Inferencing** | **Inferencing** | | [qtimlsnpe](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlsnpe.html) |

  1. Loads the segmentation model.


  2. Modifies the graph for the chosen delegate.


  3. Receives the tensor stream on its sinkpad.


  4. Runs the inference and produces a tensor stream with the segmentation results on its source pad.


| | **Postprocessing** | **Postprocessing** | | qtimlpostprocess |

  1. Receives the inference tensors on its sinkpad.


  2. Converts the inference tensors into video formats that the multimedia plugins can process later.


  3. Produces the semantic segmentations for the frame.


  4. Loads the corresponding modules for the segmentation models.


    In this use case, qtimlpostprocess does the following:




    1. Loads deeplab-argmax submodule.


    2. Produces video frames with segmentation masks.


    3. Sends them to the sinkpad of qtivcomposer.





| | [qtivcomposer](https://docs.qualcomm.com/doc/80-80021-50/topic/qtivcomposer.html) |

  1. Receives the original video stream with segmentation mask on its sinkpads.


  2. Produces on its sourcepad GST buffers with contents composed of video streams from its sinkpads.


| | [v4l2h264enc](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264enc.html) |

  1. Applies parameters to each frame of the video stream it's receiving on its sinkpad.


  2. Encodes it into bitstream and sends it over its sourcepad.


| | h264parse | Adds more information about the bitstream to the GStreamer buffer meta. | | mp4mux | Receives these buffers and creates containers format specification buffers. | | **Output** | **Output** | | Filesink | Stores the resulting stream in a */etc/media/video.mp4* file. | | Playback | Pull *video.mp4* from the host computer and play it on a media player: `scp root@\:/etc/media/video.mp4 ` | Last Published: Mar 26, 2026 [Previous Topic Image segmentation and display with Neural Processing SDK](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/single-camera-stream-with-image-segmentation-and-display-with-deeplabv3-quantized.md) [Next Topic Custom Gstreamer pipeline use cases](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/custom-gstreamer-pipeline-use-cases.md) Source: [https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-image-segmentation-and-encode-with-deeplabv3-quantized.html](https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-image-segmentation-and-encode-with-deeplabv3-quantized.html)