# Multistream batch inference
Source: [https://docs.qualcomm.com/doc/80-70022-50/topic/multistream-batch-inference.html](https://docs.qualcomm.com/doc/80-70022-50/topic/multistream-batch-inference.html)
The **gst-ai-multistream-batch-inference** application shows batched AI inference
(object detection and segmentation) on up to 24 input streams from video files.
Note: This application isn't available for the Ubuntu Server.
The following figure shows the pipeline, which receives several input streams,
preprocesses them, runs AI inferences, combines the streams with inference, and merges
them into a single video output.
The maximum number of input streams supported on each SoC are follows:
- QCS6490–8
- Dragonwing IQ-8275–4
- Dragonwing IQ-9075–4
Note: Dragonwing IQ-8275 doesn't support the Ubuntu
Server.
The output is displayed either on an HDMI display or saved as an H.264 encoded MP4
file.
For information about the plugins used in this pipeline, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70022-50/topic/multistream-batch-inference.html#multistream-batch-inference__section_lnx_1np_rcc).
Figure : gst-ai-multistream-batch-inference pipeline
## Sample model and label files
| Runtime | Model files | Label files |
| --- | --- | --- |
| LiteRT |
|
| | | |
| | | |
## Run the application on the target device
The sample application uses the
/etc/configs/config-multistream-batch-inference.json file
to read the input parameters.
To create your own config JSON file, use [config-multistream-batch-inference.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-multistream-batch-inference/config-multistream-batch-inference.json) as
a reference.
1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-70022-50/topic/download-model-and-label-files.html).
2. Update the config JSON file based on the model, input stream, and other
properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-70022-50/topic/multistream-batch-inference.html#multistream-batch-inference__section_axy_ntn_q2c).
3. Use the following format of the
`config-multistream-batch-inference.json` file:
For 16 and
24 streams, add the required elements in the `pipeline-info`
parameter. The `id` parameter takes the values from 0 to 5
for each added batch.
{
“output-type”: “wayland or filesink”,
"out-file":"",
“pipeline-info”:[
{
"id": "",
“Input type”: “”,
“input-file-path”: [
{
“”
}
],
"mlframework": "",
"model-path": "",
"labels": "",
"post processing plugin": "< qtimlvdetection or qtimlvsegmentation>"
}
]
}Copy to clipboard
For example, run the application using the LiteRT model,
video file source, label paths, and the qtimlpostprocess plugin along with
Wayland output.
{
"output-type":"wayland",
"pipeline-info":[
{
"id":0,
"input-type":"file",
"input-file-path":[
{
"stream-0":"/etc/media/video.mp4",
"stream-1":"/etc/media/video.mp4",
"stream-2":"/etc/media/video.mp4",
"stream-3":"/etc/media/video.mp4"
}
],
"mlframework":"tflite",
"model-path":"/etc/models/yolov8_det_quantized.tflite",
"labels-path":"/etc/labels/yolov8.json",
"post-process-plugin": "qtimlvdetection"
}
]
}Copy to clipboard
4. Run the gst-ai-multistream-batch-inference
application:
gst-ai-multistream-batch-inference --config-file=/etc/configs/config-multistream-batch-inference.jsonCopy to clipboard
5. To display the available help options, run the following command in the SSH
shell:
gst-ai-multistream-batch-inference -hCopy to clipboard
6. To stop the use case, use CTRL +
C.
## Expected output
Figure : Expected output for gst-ai-multistream-batch-inference–Preview
## Pipeline flow
The following table lists the plugins used in the multistream batch inference
pipeline:
| Plugin | Description |
| --- | --- |
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70022-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtibatch](https://docs.qualcomm.com/doc/80-70022-50/topic/qtibatch.html) |
Receives the batched video streams on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimltflite](https://docs.qualcomm.com/doc/80-70022-50/topic/qtimltflite.html) |
After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
|
| qtimldemux |
Demultiplexes the batched output.
Splits the output corresponding to the input streams.
|
| Postprocessing plugins | qtimlpostprocess converts the inference tensors that are received on the sink pad into video formats the multimedia plugins can use for further processing. |
| [qtivcomposer](https://docs.qualcomm.com/doc/80-70022-50/topic/qtivcomposer.html) |
Composes frames with contents from its sink pads.
Pushes the GStreamer buffers containing these composed frames to its source pad.
Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
| Filesink | Takes the video stream that it receives on its sink pad and saves it as a H.264-encoded MP4 file. |
## Config JSON field description
The different parameters available to configure the JSON file and run the use case
are as follows:
Table : Field description–gst-ai-multistream-batch-inference file
| Field | Values/description |
| --- | --- |
| **output type** | Use one of the following output types:
Wayland:Displays output on Weston.
filesink: Encodes the output in a video file.
|
| **out-file** | The file path to save the output file. |
| **pipeline -info** | Provides the pipeline information:
Stream id: Ranges from 0 to 5.
Input-type: The input source file.
Input-file-path: The array of the input file path.
|
| **mlframework** | Takes `tflite`, `qnn`, or `snpe`. |
| **model-path** | The path to the model file. |
| **labels-path** | The path to the labels file. |
## Known issues
- A drop in FPS is observed with batch size 4 models.
- Segmentation fault is observed on IQ8275 and IQ9075 using 8 stream.
## Related information
- [Object detection](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-object-detection.html)
- [Image segmentation](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-segmentation.html)
**Parent Topic:** [Run AI/ML sample applications](https://docs.qualcomm.com/doc/80-70022-50/topic/ai-ml-sample-applications.html)
Last Published: Feb 20, 2026
[Previous Topic
Multistream inference](https://docs.qualcomm.com/bundle/publicresource/80-70022-50/topics/multistream-inference.md) [Next Topic
AI smart codec](https://docs.qualcomm.com/bundle/publicresource/80-70022-50/topics/ai-smart-codec.md)