# Image classification and encode with Neural Processing SDK The use cases implement the InceptionV3 image classification model with Qualcomm Neural Processing SDK to classify scenes from a single camera stream and either overlay or compose the classification labels. The streams are then encoded. You can use any publicly available classification model with LiteRT and convert it to `.dlc` format. For instructions, see [TensorFlow Model Conversion](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-2/model_conv_tensorflow.html). ## Use qtivoverlay plugin to apply classification overlay Run the use case on the target device: gst-launch-1.0 -e filesrc location=/etc/media/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! tee name=split \ split. ! queue ! qtimetamux name=metamux ! queue ! qtivoverlay ! queue ! \ v4l2h264enc capture-io-mode=4 output-io-mode=4 ! h264parse ! mp4mux ! filesink location=/etc/media/video.mp4 \ split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/etc/ models/inceptionv3.dlc ! queue ! qtimlpostprocess \ settings="{\"confidence\": 40.0}" results=2 module=mobilenet-softmax labels=/etc/labels/classification.json ! text/x-raw ! queue ! metamux. Copy to clipboard To stop the use case, use **CTRL + C**. The following figure shows the flow of the use case execution: 1. Classify scenes from a video stream coming through a file source. 2. Overlay the classification labels using overlaylib. 3. Encode this stream as an H.264 bitstream. 4. Multiplex the stream in an MP4 container and store it as an MP4 file. Qualcomm Open source tee qtimlvconverter qtimetamux qtivoverlay v4l2h264enc h264parse mp4mux filesink qtimlsnpe qtimlpostprocess filesrc qtdemux h264parse v4l2h264dec **Figure : Pipeline for classification overlay and encode** The following table provides the sequential processing stages of the pipeline execution: | Process | Description | | --- | --- | | File source: filesrc |

  1. Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.


  2. Uses tee to split the stream for inferencing.


| | h264parse | Parses the H.264 video. | | [v4l2h264dec](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264dec.html) | Decodes the video. | | **Preprocessing** | **Preprocessing** | | [qtimlvconverter](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlvconverter.html) |

  1. Receives the video stream on its sink pad.



  2. Performs preprocessing:

    • Color conversion


    • Scaling down/up


    • Normalization on the stream data when the model expects the floating point values as input






  3. Converts the video stream to a tensor stream on its source pad.


    The classification model uses this tensor stream for inferencing.



| | **Inferencing** | **Inferencing** | | [qtimlsnpe](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlsnpe.html) |

  1. Loads the model.


  2. Modifies the graph for the chosen delegate.


  3. Receives the tensor stream on its sinkpad.


  4. Runs the inference and produces a tensor stream with the inference results on its source pad.


| | **Postprocessing** | **Postprocessing** | | qtimlpostprocess |

  1. Receives the inference tensors from a classification model on its sinkpad.


  2. Converts the tensors into formats such as video or text that the multimedia plugins can process later.


  3. Applies the threshold to the chosen number of results.


  4. Loads the corresponding modules of the classification models.


    In this use case, qtimlpostprocess does the following:




    1. Loads the submodule of the model.


    2. Produces results as video frames with classification labels.


    3. Sends them to the sinkpad of qtimetamux.





| | [qtimetamux](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimetamux.html) |

  1. Receives the video and text streams with the classification results corresponding to the video stream on its sinkpads.


  2. Produces GST buffers with the contents of the video stream on its sink pad.


  3. Adds classification result from the data sinkpad to GST buffer meta (meta muxing) on its source pad.


| | [qtivoverlay](https://docs.qualcomm.com/doc/80-80021-50/topic/qtioverlay.html) |

  1. Receives the multiplexed stream.


  2. Overlays the classification labels on the VideoFrame using CL.


  3. Produces GST buffers with overlays in its source pad.


| | [v4l2h264enc](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264enc.html) |

  1. Applies parameters to each frame of the video stream it's receiving on its sinkpad.


  2. Encodes it into bitstream and sends it over its sourcepad.


| | h264parse | Adds more information about the bitstream to GStreamer buffer meta. | | mp4mux | Receives these buffers and creates containers with format specification buffers. | | **Output** | **Output** | | Filesink | Stores the resulting stream in a */etc/media/video.mp4* file. | | Playback | Pull *video.mp4* from the host computer and play it on a media player: `scp root@\:/etc/media/video.mp4 ` | ## Use qtivcomposer to mix original frame with classification mask Run the use case on the target device: gst-launch-1.0 -e --gst-debug=2 \ filesrc location=/etc/media/Draw_1080p_18s_30FPS.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! queue ! tee name=split \ split. ! queue ! qtivcomposer name=mixer sink_1::position="<30, 30>" sink_1::dimensions="<320, 180>" ! queue ! \ v4l2h264enc capture-io-mode=4 output-io-mode=5 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/etc/media/video.mp4 \ split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/etc/ models/inceptionv3.dlc ! queue ! qtimlpostprocess settings="{\"confidence\": 40.0}" \ results=2 module=mobilenet-softmax labels=/etc/labels/classification.json ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer Copy to clipboard To stop the use case, use **CTRL + C**. The following figure shows the flow of the use case execution: - Classify scenes from a video stream coming through a file source. - Compose classification labels and video stream together using qtivcomposer. - Encode this stream as an H.264 bitstream. - Multiplex the stream in an MP4 container and store it as an MP4 file. Qualcomm Open source tee qtimlvconverter qtivcomposer v4l2h264enc h264parse mp4mux filesink qtimlsnpe qtimlpostprocess filesrc qtdemux h264parse v4l2h264dec **Figure : Pipeline for classification and encode with qtivcomposer** The following table provides the sequential processing stages of the pipeline execution: | Process | Description | | --- | --- | | File source: filesrc |

  1. Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.


  2. Uses tee to split the stream for inferencing.


| | h264parse | Parses the H.264 video. | | [v4l2h264dec](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264dec.html) | Decodes the video. | | **Preprocessing** | **Preprocessing** | | [qtimlvconverter](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlvconverter.html) |

  1. Receives the video stream on its sink pad.



  2. Performs preprocessing:

    • Color conversion


    • Scaling down/up


    • Normalization on the stream data when the model expects the floating point values as input






  3. Converts the video stream to a tensor stream on its source pad.


    The classification model uses this tensor stream for inferencing.



| | **Inferencing** | **Inferencing** | | [qtimlsnpe](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlsnpe.html) |

  1. Loads the model.


  2. Modifies the graph for the chosen delegate.


  3. Receives the tensor stream on its sinkpad.


  4. Runs the inference and produces a tensor stream with the inference results on its source pad.


| | **Postprocessing** | **Postprocessing** | | qtimlpostprocess |

  1. Receives the inference tensors from a classification model on its sinkpad.


  2. Converts the tensors into formats such as video or text that the multimedia plugins can process later.


  3. Applies the threshold to the chosen number of results.


  4. Loads the corresponding modules of the classification models.


    In this use case, qtimlpostprocess does the following:




    1. Loads the submodule of the model.


    2. Produces results as video frames with classification labels.


    3. Sends them to the sinkpad of qtivcomposer.





| | [qtivcomposer](https://docs.qualcomm.com/doc/80-80021-50/topic/qtivcomposer.html) |

  1. Receives original video stream and video stream with classification results on its sinkpads.


  2. On its sourcepad, produces GST buffers with the contents composed of video streams from its sinkpads.


| | [v4l2h264enc](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264enc.html) |

  1. Applies parameters to each frame of the video stream it's receiving on its sinkpad.


  2. Encodes it into bitstream and sends it over its sourcepad.


| | h264parse | Adds more information about the bitstream to GStreamer buffer meta. | | mp4mux | Receives these buffers and creates containers with format specification buffers. | | **Output** | **Output** | | Filesink | Stores the resulting stream in a */etc/media/video.mp4* file. | | Playback | Pull *video.mp4* from the host computer and play it on a media player: `scp root@\:/etc/media/video.mp4 ` | Last Published: Mar 26, 2026 [Previous Topic Image classification and display with Neural Processing SDK](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/single-camera-stream-with-image-classification-and-display-with-mobilenet-v1.md) [Next Topic Object detection and display with Neural Processing SDK](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/single-camera-stream-with-object-detection-and-display-with-mobilenet-v2-ssd.md) Source: [https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-image-classification-and-encode-with-mobilenet-v1.html](https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-image-classification-and-encode-with-mobilenet-v1.html)