# Image classification and display with Neural Processing SDK
Source: [https://docs.qualcomm.com/doc/80-70020-50/topic/single-camera-stream-with-image-classification-and-display-with-mobilenet-v1.html](https://docs.qualcomm.com/doc/80-70020-50/topic/single-camera-stream-with-image-classification-and-display-with-mobilenet-v1.html)
The use cases implement an Inceptionv3 model with Qualcomm Neural Processing SDK to
classify scenes, either overlay or compose the classification labels, and then display the
results.
You can use any publicly available classification model with TensorFlow and convert it to
the `.dlc` format as described in [TensorFlow Model Conversion](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-2/model_conv_tensorflow.html).
## Use qtivoverlay plugin to apply classification overlay
Run the use case on the target
device:
gst-launch-1.0 -e --gst-debug=2 \
qtiqmmfsrc name=camsrc ! video/x-raw,format=NV12_Q08C,width=1280,height=720,framerate=30/1 ! queue ! tee name=split \
split. ! queue ! qtimetamux name=metamux ! queue ! qtivoverlay ! queue ! waylandsink fullscreen=true sync=false \
split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/etc/models/inceptionv3.dlc ! queue ! \
qtimlvclassification threshold=40.0 results=2 module=mobilenet labels=/etc/labels/classification.labels ! text/x-raw ! queue ! metamux.Copy to clipboard
To stop the use case, use CTRL + C.
The following figure shows the flow of the use case execution:
- Classify scenes from a video stream coming through a camera source.
- Overlay classification labels using overlaylib.
- Display the results on a local display.
Figure : Pipeline for classification overlay
The following table provides the sequential processing stages of the pipeline
execution:
| Process | Description |
| --- | --- |
| [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70020-50/topic/qtiqmmfsrc.html) |
Collects the video stream (source) and creates two copies of the source:
One stream is sent to qtimetamux plugin to retain the video stream.
The other stream is sent to an ML inferencing pipeline.
Weston renders the video stream and possible classifications generated for that scene on a local display device.
|
## Use qtivcomposer to mix original frame with classification mask
Run the use case on the target device:
gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw,format=NV12_Q08C,width=1280,height=720,framerate=30/1 ! queue ! \
tee name=split split. ! queue ! qtivcomposer name=mixer sink_1::position="<30, 30>" sink_1::dimensions="<320, 320>" ! queue ! waylandsink fullscreen=true \
split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/etc/models/inceptionv3.dlc ! queue ! \
qtimlvclassification threshold=40.0 results=2 module=mobilenet labels=/etc/labels/classification.labels ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer.Copy to clipboard
To stop the use case, use CTRL + C.
The following figure shows the flow of the use case execution:
- Classify scenes from a video stream coming through a camera source.
- Compose classification labels and video stream together using
qtivcomposer.
- Display the results to a local display.
Figure : Pipeline for classification using qtivcomposer
The following table provides the sequential processing stages of the pipeline
execution:
| Process | Description |
| --- | --- |
| [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70020-50/topic/qtiqmmfsrc.html) |
Collects the video stream (source) and creates two copies of the source:
One stream is sent to the qtivcomposer plugin to retain the video stream.
The other stream is sent to the ML inferencing branch in the pipeline.