# Image classification and display with LiteRT The use cases use the Inceptionv3 LiteRT model to classify scenes from a single camera stream and either overlay or compose the classification labels. ## Use qtivoverlay plugin to apply classification overlay Run this use case on the target device: gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw,format=NV12_Q08C,width=1280,height=720,framerate=30/1 ! queue ! \ tee name=split split. ! queue ! qtimetamux name=metamux ! queue ! qtivoverlay ! queue ! waylandsink fullscreen=true \ split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so \ external-delegate-options="QNNExternalDelegate,backend_type=htp;" model=/etc/models/inception_v3_quantized.tflite ! queue ! \ qtimlpostprocess settings="{\"confidence\": 40.0}" results=2 module=mobilenet-softmax labels=/etc/labels/classification.json ! text/x-raw ! queue ! metamux. Copy to clipboard To stop the use case, use **CTRL + C**. The following figure shows the flow of the use case execution: 1. Classify scenes from a video stream coming through a camera source. 2. Overlay the classification labels using overlaylib. 3. Display the results. **Figure : Pipeline for classification overlay** The following table provides the sequential processing stages of the pipeline execution: | Process | Description | | --- | --- | | [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-80021-50/topic/qtiqmmfsrc.html) |
Collects the video stream (source) and creates two copies of the source:
One stream is sent to the qtimetamux plugin to retain the video stream.
The other stream is sent to an ML inferencing pipeline.
Receives the video stream on its sink pad.
Performs preprocessing:
Color conversion
Scaling down/up
Normalization on the stream data when the model expects the floating point values as an input
Converts the video stream to a tensor stream on its source pad.
The classification model uses this tensor stream for inferencing.
Loads the model.
Modifies the graph for the chosen delegate.
Receives the tensor stream on its sinkpad.
Runs the inference and produces a tensor stream with the inference results on its source pad.
Receives the inference tensors from a classification model on its sinkpad.
Converts the tensors into formats such as video or text that the multimedia plugins can process later.
Applies the threshold to the chosen number of results.
Loads the corresponding modules of the classification models.
In this use case, qtimlpostprocess does the following:
Loads the submodule of the model.
Produces results as structures of text.
Sends them to the sinkpad of qtimetamux.
Receives the video stream and text stream with classification results corresponding to the video stream on its sinkpads.
Produces GST buffers with the contents of video stream on its sink pad.
Adds classification result from data sinkpad to GST buffer meta (meta muxing) on its source pad.
Receives the multiplexed stream.
Overlays the classification labels on the VideoFrame using CL.
Produces GST buffers with overlays in its source pad.
Receives the video stream on its sinkpad.
Submits the video stream to Weston.
Weston renders the video stream and possible classifications generated for that scene on a local display device.
Collects the video stream (source) and creates two copies of the source:
One stream is sent to the qtivcomposer plugin to retain the video stream.
The other stream is sent to the ML inferencing branch in the pipeline.
Receives the video stream on its sink pad.
Color conversion
Scaling down/up
Normalization on the stream data when the model expects the floating point values as an input
Converts the video stream to a tensor stream on its source pad.
The classification model uses this tensor stream for inferencing.
Loads the model.
Modifies the graph for the chosen delegate.
Receives the tensor stream on its sinkpad.
Runs the inference and produces a tensor stream with the inference results on its source pad.
Receives the inference results from a classification model on its sinkpad.
Converts the inference tensors into formats like video or text that the multimedia plugins can process later.
Applies the threshold to the chosen number of results.
Loads the corresponding modules for the classification models.
In this use case, qtimlpostprocess does the following:
Loads the submodule of the model.
Produces results as video frames with classification labels.
Sends them to the sinkpad of qtivcomposer.
Receives the original video stream with classification results on its sinkpads.
On its sourcepad, produces GST buffers with contents composed of video streams from its sinkpads.
Receives the video in its sinkpad
Submits the video stream to Weston.
Weston renders the video stream and possible classifications generated for that scene on a local display device.