# Multistream inference
The **gst-ai-multistream-inference** application shows AI inference (object detection and classification) on up to 32 input streams coming from camera, file, or RTSP stream.
The following figure shows the pipeline, which receives several input streams, preprocesses them, runs AI inferences, combines the streams, and merges them all into a single video output.
The maximum number of input streams supported on each SoC as verified on 1080P and 720P are follows:
- QCS6490–8
- Dragonwing IQ-8275–16
- Dragonwing IQ-9075–32
Note
This application isn't supported in Config #1 for the QLI 2.0 RC3 release because CPU runtime is not supported.
Note
Dragonwing IQ-9075 isn't supported on the Ubuntu Server.
The output is displayed on an HDMI display, saved as an H.264 encoded MP4 file, or converted into an RTSP stream.
For information about the plugins used in this pipeline, see [Pipeline flow](https://docs.qualcomm.com/doc/80-80022-55/topic/multistream-inference.html#section-opk-tn2-ncc).
Qualcomm
Open source
File (default)
Stream 1
Camera (optional)
RTSP (optional)
qtivcomposer
tee
tee
qtdemux
h264parse
V4l2h264dec
qtimlvconverter
qtimltflite/
qtimlpostprocess
filesrc
or
or
waylandsink
filesink
qtirtspbin
qtivcomposer
qtimlvconverter
qtimltflite
qtimlpostprocess
qtiqmmfsrc
or
or
waylandsink
filesink
qtirtspbin
qtivcomposer
rtph264 depay
h264parse
V4l2h264dec
tee
qtimlvconverter
qtimltflite
qtimlpostprocess
rtspsrc
or
or
waylandsink
filesink
qtirtspbin
File (default)
Stream 32
Camera (optional)
RTSP (optional)
qtivcomposer
tee
tee
qtdemux
h264parse
V4l2h264dec
qtimlvconverter
qtimltflite/
qtimlpostprocess
filesrc
or
or
waylandsink
filesink
qtirtspbin
qtivcomposer
qtimlvconverter
qtimltflite
qtimlpostprocess
qtiqmmfsrc
or
or
waylandsink
filesink
qtirtspbin
qtivcomposer
rtph264 depay
h264parse
V4l2h264dec
tee
qtimlvconverter
qtimltflite
qtimlpostprocess
rtspsrc
or
or
waylandsink
filesink
qtirtspbin
Stream 2
Stream 31
**Figure : gst-ai-multistream-inference pipeline**
## Input and output capabilities
The following table summarizes the input and output capabilities supported by the sample application:
| Config | Input | Input | Input | Input | Output | Output | Output |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Config | File src | RTSP | USB camera | MIPI camera | File | Display | RTSP |
| Config #2 | Yes | Yes | No | Yes | Yes | Yes | Yes |
| | | | | | | | |
## Sample model and label files
Table : Sample model and label files for gst-ai-multistream-inference
| Runtime | Model files | Label files |
| --- | --- | --- |
| LiteRT |
| yolox.json
classification.json
|
## Run the application on the target device
The sample application uses the `/etc/configs/config-multistream-inference.json` file to read the input parameters.
To create your own config JSON file, use [config-multistream-inference.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-multistream-inference/config-multistream-inference.json) as a reference.
1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-80022-55/topic/download-model-and-label-files.html).
2. Update the config JSON file based on the model, input stream, and other properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-80022-55/topic/multistream-inference.html#section-x4t-bj1-42c).
3. Use the following format of the `config-multistream-inference.json` file:
{
"input-file-path": [
" ",
" "
],
"input-rtsp-path": [
"",
""
],
"input-type": "",
"model": "",
"labels": "",
"output-display": "<0 or 1>",
"output-file-path": "",
"output-ip-address": "",
"output-port-number": "",
"use-case": "<0 or 1>"
}
Copy to clipboard
For example, run the application for object detection on 8 H.264 file inputs and view the output on a Wayland display:
{
"input-file-path":
[
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4"
],
"model": "/etc/models/yolox_quantized.tflite",
"labels": "/etc/labels/yolox.json",
"input-type": "h264",
"output-display": 1,
"use-case": 0
}
Copy to clipboard
4. Run the gst-ai-multistream-inference application:
gst-ai-multistream-inference --config-file=/etc/configs/config-multistream-inference.json
Copy to clipboard
Note
If a drop in performance is observed, you can use YOLOv8 LiteRT model. For YOLOv8 export instructions, see [Prerequisites](https://docs.qualcomm.com/doc/80-80022-55/topic/download-model-and-label-files.html).
5. To display the available help options, run the following commands in the SSH shell:
gst-ai-multistream-inference -h
Copy to clipboard
6. To stop the use case, use **CTRL + C**.
## Expected output

**Figure : Expected output for gst-ai-multistream-inference**
## Pipeline flow
The following table lists the plugins used in the multistream inference pipeline:
| Plugin | Description |
| --- | --- |
| Camera source: [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-80022-50/topic/qtiqmmfsrc.html) | |
| File source: filesrc | Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc | Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-80022-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-80022-50/topic/qtimlvconverter.html) | Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimltflite](https://docs.qualcomm.com/doc/80-80022-50/topic/qtimltflite.html) | It runs on LiteRT and uses a quantized YOLOv8 model for object detection and a quantized Inception v3 model for classification.After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
|
| [qtimlpostprocess](https://docs.qualcomm.com/doc/80-80022-50/topic/qtimlpostprocess.html) | Handles the inference results from any object detection, classification, pose detection, and segmentation model. |
| [qtivcomposer](https://docs.qualcomm.com/doc/80-80022-50/topic/qtivcomposer.html) | Composes frames with contents from its sink pads.
Pushes the GStreamer buffers containing these composed frames to its source pad.
|
| [waylandsink](https://docs.qualcomm.com/doc/80-80022-50/topic/waylandsink.html) | Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
| Filesink | Receives the video stream on its sink pad.
Saves the stream as a H.264-encoded MP4 file.
|
| [qtirtspbin](https://docs.qualcomm.com/doc/80-80022-50/topic/qtirtspbin.html) | Serves as a network sink.
Transmits UDP packets to the network.
|
## Config JSON field description
The different parameters available to configure the JSON file and run the use case are as follows:
Table : Field description–gst-ai-multistream-inference file
| Field | Values/description |
| --- | --- |
| **Input source** | Use one of the following input sources:num-camera : The number of inputs from the camera.
camera : The input camera if num-camera=1 .
input-file-path : The directory path to the video file.
input-rtsp-path : The address of the RTSP stream: rtsp://<ip>:<port>/<stream>
|
| **input-type** | The video encoding type for file and RTSP input: |
| **Output** | Use one of the following outputs: \* *output-file-path*: The directory path to save the output file. \* *output-ip-address*: The IP address of the device on which the RTSP stream can be played. \* *output-port-number*: The port number of the device on which the RTSP stream can be played. \* *output-display*: The connected display device for preview. Select 1 to enable this option. |
## Known issue
>
>
> Low FPS and frame drop is observed during inference in Config #2 on Dragonwing IQ‑8275, Dragonwing IQ‑9075, and QCS6490.
## Related information
- [Image classification](https://docs.qualcomm.com/doc/80-80022-55/topic/gst-ai-classification.html)
- [Object detection](https://docs.qualcomm.com/doc/80-80022-55/topic/gst-ai-object-detection.html)
Last Published: May 17, 2026
[Previous Topic
Video super-resolution](https://docs.qualcomm.com/bundle/publicresource/80-80022-55/topics/video-super-resolution.md) [Next Topic
Multistream batch inference](https://docs.qualcomm.com/bundle/publicresource/80-80022-55/topics/multistream-batch-inference.md)
Source: [https://docs.qualcomm.com/doc/80-80022-55/topic/multistream-inference.html](https://docs.qualcomm.com/doc/80-80022-55/topic/multistream-inference.html)