# Image classification and display with ONNX

The use cases use an ONNX model to perform scene classification within a single‑camera media pipeline, with inference results overlaid on or composed into the output stream.

Run this use case on the target device:

export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1 && \
    gst-launch-1.0 -v filesrc location=/etc/media/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12 ! tee name=split \
    split. ! queue ! qtivcomposer name=mixer ! queue ! waylandsink fullscreen=true \
    split. ! queue ! qtimlvconverter ! queue ! qtimlonnx model=/etc/models/model.onnx execution-provider=qnn backend-path="/usr/lib/libQnnHtp.so" ! queue ! qtimlpostprocess results=1 module=mobilenet-softmax labels=/etc/labels/classification.json settings="{\"confidence\": 51.0}" ! video/x-raw,format=BGRA,width=640,height=360 ! queue ! mixer.
    Copy to clipboard

Note

When using an `ONNXModel`, place the model weight file in the same directory as the ONNX model file and name it `model.data`.

To stop the use case, use **CTRL + C**.

The following figure shows the flow of the use case execution:

1. Classify scenes from a video stream coming through a camera source.
2. Overlay the classification labels using overlaylib.
3. Display the results.

<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="755" height="349.974590301513672" viewbox="0 0 755 349.974590301513672" aria-label="../../_images/image_classification_onnx.svg">
  <defs>
    <style>.svg-1 .cls-1 { fill: none; stroke: #000; stroke-miterlimit: 10 }
.svg-1 .cls-2 { fill: #fff; font-size: 16px }
.svg-1 .cls-2,.svg-1 .cls-3 { font-family: Roboto-Regular, Roboto }
.svg-1 .cls-4 { fill: #007884 }
.svg-1 .cls-5 { fill: #d2d7e1 }
.svg-1 .cls-6 { fill: #2a2aea }
.svg-1 .cls-3 { font-size: 14px }
.svg-1 .cls-7 { fill: #fafafa }</style>
  </defs>
  <g id="Layer_1" data-name="Layer 1">
    <g>
      <rect class="cls-7" x=".5" y=".499797821044922" width="754" height="348.974609375" rx="7.499999999999944" ry="7.499999999999944"></rect>
      <path class="cls-5" d="M747,1c3.85980224609375,0,7,3.140201568603516,7,7v333.974590301513672c0,3.85980224609375-3.14019775390625,7-7,7H8c-3.85980224609375,0-7-3.14019775390625-7-7V8c0-3.859798431396484,3.14019775390625-7,7-7h739M747,0H8C3.581695556640625,0,0,3.581701278686523,0,8v333.974590301513672c0,4.418304443359375,3.581695556640625,8,8,8h739c4.41827392578125,0,8-3.581695556640625,8-8V8c0-4.418298721313477-3.58172607421875-8-8-8h0Z"></path>
    </g>
    <g>
      <g>
        <text class="cls-3" transform="translate(555.4927978515625 326.066066741943359)"><tspan x="0" y="0">Qualcomm </tspan></text>
        <rect class="cls-6" x="535.241806286625433" y="313.974590301513672" width="16" height="16" rx="2" ry="2"></rect>
      </g>
      <g>
        <text class="cls-3" transform="translate(654.0745849609375 326.066066741943359)"><tspan x="0" y="0">Open source</tspan></text>
        <rect class="cls-4" x="633.823587699482232" y="313.974590301513672" width="16" height="16" rx="2" ry="2"></rect>
      </g>
    </g>
  </g>
  <g id="Layer_2" data-name="Layer 2">
    <g>
      <g>
        <rect class="cls-4" x="20" y="20.000014873722648" width="160" height="50" rx="4" ry="4"></rect>
        <text class="cls-2" transform="translate(73.429729461669922 48.506718635559082)"><tspan x="0" y="0">camsrc</tspan></text>
      </g>
      <g>
        <line class="cls-1" x1="180" y1="45.000011444091797" x2="199.976654052734375" y2="45.000011444091797"></line>
        <polygon points="198.955352783203125 48.490444183349609 205 45.000011444091797 198.955352783203125 41.50958251953125 198.955352783203125 48.490444183349609"></polygon>
      </g>
      <g>
        <line class="cls-1" x1="285" y1="68.974590301513672" x2="285" y2="88.951229095458984"></line>
        <polygon points="281.50958251953125 87.929935455322266 285 93.974590301513672 288.49041748046875 87.929935455322266 281.50958251953125 87.929935455322266"></polygon>
      </g>
      <g>
        <rect class="cls-4" x="205" y="20.000014873722648" width="160" height="50" rx="4" ry="4"></rect>
        <text class="cls-2" transform="translate(273.910202026367188 48.506718635559082)"><tspan x="0" y="0">tee</tspan></text>
      </g>
      <g>
        <rect class="cls-6" x="205" y="93.97459047333632" width="160" height="50" rx="4" ry="4"></rect>
        <text class="cls-2" transform="translate(229.527496337890625 123.650547027587891)"><tspan x="0" y="0">qtimlvconverter</tspan></text>
      </g>
      <g>
        <line class="cls-1" x1="285" y1="143.974590301513672" x2="285" y2="163.951229095458984"></line>
        <polygon points="281.50958251953125 162.929943084716797 285 168.974590301513672 288.49041748046875 162.929943084716797 281.50958251953125 162.929943084716797"></polygon>
      </g>
      <g>
        <rect class="cls-6" x="205" y="168.97459047333632" width="160" height="50" rx="4" ry="4"></rect>
        <text class="cls-2" transform="translate(249.589996337890625 198.650554656982422)"><tspan x="0" y="0">qtimlonnx</tspan></text>
      </g>
      <g>
        <line class="cls-1" x1="285" y1="218.974590301513672" x2="285" y2="238.951244354248047"></line>
        <polygon points="281.50958251953125 237.929943084716797 285 243.974590301513672 288.49041748046875 237.929943084716797 281.50958251953125 237.929943084716797"></polygon>
      </g>
      <g>
        <rect class="cls-6" x="205" y="243.97459047333632" width="160" height="49.999999999999091" rx="4" ry="4"></rect>
        <text class="cls-2" transform="translate(222.722808837890625 273.650554656982422)"><tspan x="0" y="0">qtimlpostprocess</tspan></text>
      </g>
      <g>
        <line class="cls-1" x1="365" y1="45.000011444091797" x2="384.97662353515625" y2="45.000011444091797"></line>
        <polygon points="383.955322265625 48.490444183349609 390 45.000011444091797 383.955322265625 41.50958251953125 383.955322265625 48.490444183349609"></polygon>
      </g>
      <g>
        <rect class="cls-6" x="390" y="20.000014873722648" width="160" height="50" rx="4" ry="4"></rect>
        <text class="cls-2" transform="translate(421.140655517578125 48.506718635559082)"><tspan x="0" y="0">qtivcomposer</tspan></text>
      </g>
      <g>
        <line class="cls-1" x1="550" y1="45.000011444091797" x2="569.97662353515625" y2="45.000011444091797"></line>
        <polygon points="568.955322265625 48.490444183349609 575 45.000011444091797 568.955322265625 41.50958251953125 568.955322265625 48.490444183349609"></polygon>
      </g>
      <g>
        <rect class="cls-4" x="575" y="20.000014873722648" width="160" height="50" rx="4.000000000000019" ry="4.000000000000019"></rect>
        <text class="cls-2" transform="translate(611.09771728515625 48.506718635559082)"><tspan x="0" y="0">waylandsink</tspan></text>
      </g>
    </g>
    <g>
      <polyline class="cls-1" points="365 268.974590301513672 470 268.974590301513672 470 75.847972869873047"></polyline>
      <polygon points="473.49041748046875 76.869258880615234 470 70.824611663818359 466.50958251953125 76.869258880615234 473.49041748046875 76.869258880615234"></polygon>
    </g>
  </g>
</svg>
**Figure : Pipeline for classification overlay**

The following table provides the sequential processing stages of the pipeline execution:

**Table : Pipeline processing stages for image classification **

| Process | Description |
| --- | --- |
| qtiqmmfsrc | <ol class="arabic"><br><li><p>Collects the video stream (source) and creates two copies of the source:</p><br><blockquote><br><div><ul class="simple"><br><li><p>One stream is sent to the qtimetamux plugin to retain the video stream.</p></li><br><li><p>The other stream is sent to an ML inferencing pipeline.</p></li><br></ul><br></div></blockquote><br></li><br></ol> |
| **Preprocessing** |
| qtimlvconverter | <ol class="arabic"><br><li><p>Receives the video stream on its sink pad.</p></li><br><li><p>Performs preprocessing:</p><br><blockquote><br><div><ul class="simple"><br><li><p>Color conversion</p></li><br><li><p>Scaling down/up</p></li><br><li><p>Normalization on the stream data when the model expects the floating point values as an input</p></li><br></ul><br></div></blockquote><br></li><br><li><p>Converts the video stream to a tensor stream on its source pad.</p><br><p>The classification model uses this tensor stream for inferencing.</p><br></li><br></ol> |
| **Inferencing** |
| qtimlonnx | <ol class="arabic simple"><br><li><p>Loads the model.</p></li><br><li><p>Modifies the graph for the chosen delegate.</p></li><br><li><p>Receives the tensor stream on its sinkpad.</p></li><br><li><p>Runs the inference and produces a tensor stream with the inference results on its source pad.</p></li><br></ol> |
| **Postprocessing** |
| qtimlpostprocess | <ol class="arabic"><br><li><p>Receives the inference tensors from a classification model on its sinkpad.</p></li><br><li><p>Converts the tensors into formats such as video or text that the multimedia plugins can process later.</p></li><br><li><p>Applies the threshold to the chosen number of results.</p></li><br><li><p>Loads the corresponding modules of the classification models.</p><br><p>In this use case, qtimlpostprocess does the following:</p><br><blockquote><br><div><ol class="loweralpha simple"><br><li><p>Loads the submodule of the model.</p></li><br><li><p>Produces results as structures of text.</p></li><br><li><p>Sends them to the sinkpad of qtimetamux.</p></li><br></ol><br></div></blockquote><br></li><br></ol> |
| qtimetamux | <ol class="arabic simple"><br><li><p>Receives the video stream and text stream with classification results corresponding to the video stream on its sinkpads.</p></li><br><li><p>Produces GST buffers with the contents of video stream on its sink pad.</p></li><br><li><p>Adds classification result from data sinkpad to GST buffer meta (meta muxing) on its source pad.</p></li><br></ol> |
| qtivoverlay | <ol class="arabic simple"><br><li><p>Receives the multiplexed stream.</p></li><br><li><p>Overlays the classification labels on the VideoFrame using CL.</p></li><br><li><p>Produces GST buffers with overlays in its source pad.</p></li><br></ol> |
| **Output** |
| Waylandsink | <ol class="arabic simple"><br><li><p>Receives the video stream on its sinkpad.</p></li><br><li><p>Submits the video stream to Weston.</p></li><br><li><p>Weston renders the video stream and possible classifications generated for that scene on a local display device.</p></li><br></ol> |

Last Published: Apr 02, 2026

Previous Topic
 
ONNX use cases Next Topic

Object detection with ONNX