# Multistream batch inference
Source: [https://docs.qualcomm.com/doc/80-70020-50/topic/multistream-batch-inference.html](https://docs.qualcomm.com/doc/80-70020-50/topic/multistream-batch-inference.html)
The **gst-ai-multistream-batch-inference** application shows batched AI inference
(object detection and segmentation) on up to 24 input streams from video files.
The following figure shows the pipeline, which receives several input streams,
preprocesses them, runs AI inferences, combines the streams with inference, and merges
them into a single video output.
The maximum number of input streams supported on each SoC are follows:
- QCS6490–8
- Dragonwing IQ-8275–4
- Dragonwing IQ-9075–4
Note: For Ubuntu Server, this sample application isn't
supported on Dragonwing IQ-8275.
The output is displayed either on an HDMI display or saved as an H.264 encoded MP4
file.
For information about the plugins used in this pipeline, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70020-50/topic/multistream-batch-inference.html#multistream-batch-inference__section_lnx_1np_rcc).
Figure : gst-ai-multistream-batch-inference pipeline
## Sample model and label files
| Runtime | Model files | Label files |
| --- | --- | --- |
| LiteRT |
|
| | | |
| | | |
## Run the application on the target device
The sample application uses the
/etc/configs/config-multistream-batch-inference.json file
to read the input parameters
To create your own config JSON file, use [config-multistream-batch-inference.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-multistream-batch-inference/config-multistream-batch-inference.json) as
a reference.
1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-70020-50/topic/download-model-and-label-files.html).
2. Update the config JSON file based on the model, input stream, and other
properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-70020-50/topic/multistream-batch-inference.html#multistream-batch-inference__section_axy_ntn_q2c)
Note: In the
config-multistream-batch-inference.jsonfile, with
the batch size as 4, you can replace the following:
- model-batch-4.tflite with
`yolov8_det_quantized.tflite`
- detection.labels with
`yolov8.labels`
- constants with
`"yolov8,q-offsets=<24.0, 0.0,
0.0>,q-scales=<3.146998405456543, 0.003778052283450961,
1.0>;"`
3. Use the following format of the
`config-multistream-batch-inference.json` file:
For 16 and
24 streams, add the required elements in the `pipeline-info`
parameter. The `id` parameter takes the values from 0 to 5
for each added batch.
{
“output-type”: “wayland or filesink”,
"out-file":"",
“pipeline-info”:[
{
"id": "",
“Input type”: “”,
“input-file-path”: [
{
“”
}
],
"mlframework": "",
"model-path": "",
"labels": "",
"constants": "",
"post processing plugin": "< qtimlvdetection or qtimlvsegmentation>"
}
]
}
Copy to clipboard
For example, run the application using the LiteRT model,
video file source, label paths, and the qtimlvdetection postprocessing
plugin along with Wayland
output.
{
"output-type":"wayland",
"pipeline-info":[
{
"id":0,
"input-type":"file",
"input-file-path":[
{
"stream-0":"/etc/media/video.mp4",
"stream-1":"/etc/media/video.mp4",
"stream-2":"/etc/media/video.mp4",
"stream-3":"/etc/media/video.mp4"
}
],
"mlframework":"tflite",
"model-path":"/etc/models/yolov8_det_quantized.tflite",
"labels-path":"/etc/labels/yolov8.labels",
"constants":"yolov8,q-offsets=<24.0, 0.0, 0.0>,q-scales=<3.146998405456543, 0.003778052283450961, 1.0>;",
"post-process-plugin": "qtimlvdetection"
}
]
}Copy to clipboard
4. Run the gst-ai-multistream-batch-inference
application:
gst-ai-multistream-batch-inference --config-file=/etc/configs/config-multistream-batch-inference.jsonCopy to clipboard
5. To display the available help options, run the following command in the SSH
shell:
gst-ai-multistream-batch-inference -hCopy to clipboard
6. To stop the use case, use CTRL +
C.
## Expected output
Figure : Expected output for gst-ai-multistream-batch-inference–Preview
## Pipeline flow
The following table lists the plugins used in the multistream batch inference
pipeline:
| Plugin | Description |
| --- | --- |
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70020-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtibatch](https://docs.qualcomm.com/doc/80-70020-50/topic/qtibatch.html) |
Receives the batched video streams on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimltflite](https://docs.qualcomm.com/doc/80-70020-50/topic/qtimltflite.html) |
After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
|
| qtimldemux |
Demultiplexes the batched output.
Splits the output corresponding to the input streams.
|
| Postprocessing plugins | [qtimlvsegmentation](https://docs.qualcomm.com/doc/80-70020-50/topic/qtimlvsegmentation.html) and [qtimlvdetection](https://docs.qualcomm.com/doc/80-70020-50/topic/qtimlvdetection.html): Convert the inference tensors that are received on the sink pad into video formats the multimedia plugins can use for further processing. |
| [qtivcomposer](https://docs.qualcomm.com/doc/80-70020-50/topic/qtivcomposer.html) |
Composes frames with contents from its sink pads.
Pushes the GStreamer buffers containing these composed frames to its source pad.
Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
| Filesink | Takes the video stream that it receives on its sink pad and saves it as a H.264-encoded MP4 file. |
## Config JSON field description
The different parameters available to configure the JSON file and run the use case
are as follows:
Table : Field description–gst-ai-multistream-batch-inference file
| Field | Values/description |
| --- | --- |
| **output type** | Use one of the following output type:
Wayland:Displays output on Weston.
filesink: Encodes the output in a video file.
|
| **out-file** | The file path to save the output file. |
| **pipeline -info** | Provides the pipeline information:
Stream id: Ranges from 0 to 5
Input-type: The input source file.
Input-file-path: The array of the input file path.
|
| **mlframework** | Takes `tflite`, `qnn`, or `snpe`. |
| **model-path** | The path to the model file. |
| **labels-path** | The path to the labels file. |
| **constants** | The constant of the LiteRT model. |
## Known issues
- Frame freeze or segmentation fault is observed some time after starting the
application.
- Misplaced bounding box is observed with Yolov8 SNPE model.
## Related information
- [Object detection](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-object-detection.html)
- [Image segmentation](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-segmentation.html)
**Parent Topic:** [Run AI/ML sample applications](https://docs.qualcomm.com/doc/80-70020-50/topic/ai-ml-sample-applications.html)
Last Published: Jan 30, 2026
[Previous Topic
Multistream inference](https://docs.qualcomm.com/bundle/publicresource/80-70020-50/topics/multistream-inference.md) [Next Topic
AI smart codec](https://docs.qualcomm.com/bundle/publicresource/80-70020-50/topics/ai-smart-codec.md)