# Multistream batch inference
Source: [https://docs.qualcomm.com/doc/80-70018-50/topic/multistream-batch-inference.html](https://docs.qualcomm.com/doc/80-70018-50/topic/multistream-batch-inference.html)
The **gst-ai-multistream-batch-inference** application shows batched AI inference
(object detection and segmentation) on up to 24 input streams from video files.
Note: The maximum number of input streams supported on each SoC are
follows:
- QCS6490–8
- QCS8275–4
- QCS9075–4
The figure shows the pipeline, which takes several input streams, does the preprocessing
and AI inferencing stream on each video stream, combines the streams with inference, and
merges them into a single video output.
The output is displayed either on an HDMI display or saved as an H.264 encoded MP4
file.
For information about the plugins used in this pipeline, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70018-50/topic/multistream-batch-inference.html#multistream-batch-inference__section_lnx_1np_rcc).
Figure : gst-ai-multistream-batch-inference pipeline
## Sample model and label files
Table : Sample model and label files for gst-ai-multistream-batch-inference
| Runtime | Model files | Label files |
| :--- | :--- | :--- |
| LiteRT |
|
## Prerequisites
- If not already done so, [Download and install eSDK](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-51/install-sdk.html#download-and-install-esdk-).
- To run the application, push the model files.
The application supports the
deeplabv3\_plus\_mobilenet\_quantized.tflite and
yolov8\_det\_quantized.tflite models for
batching.
- To generate the batching model, see [Download model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html).
- For model files, see [Sample model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/multistream-batch-inference.html#multistream-batch-inference__section_ik5_xqh_4dc).
Note: The gst-ai-multistream-batch-inference application uses the
`batch-size=4` model. To change the batch size of the
model, update the `export`
command:
python -m qai_hub_models.models..export --quantize w8a8 --batch-size --device "QCS6490 (Proxy)"Copy to clipboard
- To access your host device, enable SSH. For instructions, see [Sign in using SSH](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-254/how_to.html#use-ssh).
Note: If SSH is already enabled, you can skip this
step.
- Connect the display to the device using the HDMI port. For instructions, see
[Set up HDMI display](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/samples.html).
- Enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
- Push the model files from the Linux host
computer:
scp root@:/etc/modelsCopy to clipboard
- Export the `cdsp1` library
path:
export ADSP_LIBRARY_PATH="/usr/lib/dsp/cdsp;/usr/lib/dsp/cdsp1;$ADSP_LIBRARY_PATH"Copy to clipboard
If you face issues while enabling camera or display, see [Camera troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-17/troubleshooting.html) and [Display troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/debug.html).
## Run the application
The sample application uses the
/etc/configs/config-multistream-batch-inference.json file
to read the input parameters
To create your own config JSON file, use [config-multistream-batch-inference.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/tree/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-multistream-batch-inference?ref_type=heads) as
a reference.
1. Use the following format of the
`config-multistream-batch-inference.json` file:
For 16 and
24 streams, add the required elements in the `pipeline-info`
parameter. The `id` parameter takes the values from 0 to 5
for each added batch.
{
“output-type”: “wayland or filesink”,
"out-file":"",
“pipeline-info”:[
{
"id": "",
“Input type”: “”,
“input-file-path”: [
{
“”
}
],
"mlframework": "",
"model-path": "",
"labels": "",
"constants": "",
"post processing plugin": "< qtimlvdetection or qtimlvsegmentation>"
}
]
}
Copy to clipboard
Note: Update the config JSON file based on
the model, input stream, and other properties. For more information, see
[Config JSON field description](https://docs.qualcomm.com/doc/80-70018-50/topic/multistream-batch-inference.html#multistream-batch-inference__section_axy_ntn_q2c).
In
the config-multistream-batch-inference.json file,
with the batch size as 4, you can replace the following:
- model-batch-4.tflite with
`yolov8_det_quantized.tflite`
- detection.labels with
`yolov8.labels`
- constants with
`"yolov8,q-offsets=<24.0, 0.0,
0.0>,q-scales=<3.146998405456543, 0.003778052283450961,
1.0>;"`
For example, run the application using the LiteRT model,
video file source, label paths, and the qtimlvdetection postprocessing
plugin along with Wayland
output.
{
"output-type":"wayland",
"pipeline-info":[
{
"id":0,
"input-type":"file",
"input-file-path":[
{
"stream-0":"/etc/media/video.mp4",
"stream-1":"/etc/media/video.mp4",
"stream-2":"/etc/media/video.mp4",
"stream-3":"/etc/media/video.mp4"
}
],
"mlframework":"qtimltflite",
"model-path":"/etc/models/yolov8_det_quantized.tflite",
"labels-path":"/etc/labels/yolov8.labels",
"constants":"yolov8,q-offsets=<24.0, 0.0, 0.0>,q-scales=<3.146998405456543, 0.003778052283450961, 1.0>;",
"post-process-plugin": "qtimlvdetection"
}
]
}Copy to clipboard
2. Run the gst-ai-multistream-batch-inference
application:
gst-ai-multistream-batch-inference --config-file=/etc/configs/config-multistream-batch-inference.jsonCopy to clipboard
To display the available help options, run the following command in the SSH
shell:
gst-ai-multistream-batch-inference -hCopy to clipboard
To stop the use case, use CTRL + C.
## Expected output
Figure : Expected output for gst-ai-multistream-batch-inference–Preview
## Pipeline flow
The table lists the plugins used in the multistream batch inference
pipeline:| Plugin | Description |
| --- | --- |
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70018-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtibatch](https://docs.qualcomm.com/doc/80-70018-50/topic/qtibatch.html) |
Receives the batched video streams on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimltflite](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimltflite.html) |
After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
|
| qtimldemux |
Demultiplexes the batched output.
Splits the output corresponding to the input streams.
|
| Postprocessing plugins | [qtimlvsegmentation](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvsegmentation.html) and [qtimlvdetection](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvdetection.html): Convert the inference tensors that are received on the sink pad into video formats the multimedia plugins can use for further processing. |
| [qtivcomposer](https://docs.qualcomm.com/doc/80-70018-50/topic/qtivcomposer.html) |
Composes frames with contents from its sink pads.
Pushes the GStreamer buffers containing these composed frames to its source pad.
Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
| Filesink | Takes the video stream that it receives on its sink pad and saves it as a H.264-encoded MP4 file. |
## Config JSON field description
Table : Field description–gst-ai-multistream-batch-inference file
| Field | Values/description |
| --- | --- |
| **output type** | Use one of the following output type:
Wayland:Displays output on Weston.
filesink: Encodes the output in a video file.
|
| **out-file** |
The file path to save the output file.
|
| **pipeline -info** | Provides the pipeline information:
Stream id: Ranges from 0 to 5
Input-type: The input source file.
Input-file-path: The array of the input file path.
|
| **mlframework** | Uses the inference plugin such as qtimltflite. |
| **model-path** | The path to the model file. |
| **labels-path** | The path to the labels file. |
| **constants** | The constant of the LiteRT model. |
## Known issues
Frame freeze is observed some time after starting the application.
**Parent Topic:** [Run AI/ML sample applications](https://docs.qualcomm.com/doc/80-70018-50/topic/ai-ml-sample-applications.html)
**Related Resources**
- [Object detection](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-object-detection.html)
- [Image segmentation](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-segmentation.html)
Last Published: Jan 30, 2026
[Previous Topic
Multistream inference](https://docs.qualcomm.com/bundle/publicresource/80-70018-50/topics/multistream-inference.md) [Next Topic
AI smart codec](https://docs.qualcomm.com/bundle/publicresource/80-70018-50/topics/ai-smart-codec.md)