# Multistream inference
Source: [https://docs.qualcomm.com/doc/80-70020-50/topic/multistream-inference.html](https://docs.qualcomm.com/doc/80-70020-50/topic/multistream-inference.html)
The **gst-ai-multistream-inference** application shows AI inference (object
detection and classification) on up to 32 input streams coming from camera, file, or RTSP
stream.
The following figure shows the pipeline, which receives several input streams,
preprocesses them, runs AI inferences, combines the streams, and merges them all into a
single video output.
The maximum number of input streams supported on each SoC are follows:
- QCS6490–8
- Dragonwing IQ-8275–16
- Dragonwing IQ-9075–32
Note: For Ubuntu Server, this sample application isn't
supported on Dragonwing IQ-8275.
The output is displayed on an HDMI display, saved as an H.264 encoded MP4 file, or
converted into an RTSP stream.
For information about the plugins used in this pipeline, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70020-50/topic/multistream-inference.html#multistream-inference__section_opk_tn2_ncc).
Figure : gst-ai-multistream-inference pipeline
## Sample model and label files
Table : Sample model and label files for gst-ai-multistream-inference
| Runtime | Model files | Label files |
| :--- | :--- | :--- |
| LiteRT | inception_v3_quantized.tflite yolox_quantized.tflite yolox.labels Classification.labels ",
""
],
"input-type": ""
"model": "",
"labels": "",
"constants" : "",
"output-display": <0 or 1>,
"output-file-path": "",
"output-ip-address": "",
"output-port-number": "",
"use-case": <0 or 1>
}Copy to clipboard
For example, run the application for object detection on 16 H.264 file inputs and view the output
on a Wayland display:
{
"input-file-path":
[
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4",
"/etc/media/video.mp4"
],
"model": "/etc/models/yolox_quantized.tflite",
"labels": "/etc/labels/yolox.labels",
"constants": "YOLOx,q-offsets=<38.0, 0.0, 0.0>,q-scales=<3.6124823093414307, 0.003626860911026597, 1.0>;",
"input-type": "h264",
"output-display": 1,
"use-case": 0
}Copy to clipboard
4. Run the gst-ai-multistream-inference application:
gst-ai-multistream-inference --config-file=/etc/configs/config-multistream-inference.jsonCopy to clipboard
5. To display the available help options, run the following commands in the SSH
shell:
gst-ai-multistream-inference -hCopy to clipboard
6. To stop the use case, use CTRL + C.
## Expected output
Figure : Expected output for gst-ai-multistream-inference
## Pipeline flow
The following table lists the plugins used in the multistream inference
pipeline:| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70020-50/topic/qtiqmmfsrc.html) | Captures the live stream from camera. Uses tee to split the stream for inferencing. Captures the video stream using filesrc, followed by Uses tee to split the stream for inferencing. Captures the RTSP stream using rtspsrc, followed by Uses tee to split the stream for inferencing. Receives the video stream on its sink pad. Performs the following preprocessing on the stream data.Color conversion Scaling (up or down) Normalization After the inference runtime receives the tensor stream on Produces a tensor stream with the inference results on its qtimlvdetection :Applies a threshold to the chosen number of Loads the YOLO-NAS module. Produces video frames with only bounding boxes that qtimlvclassification Applies the threshold to the chosen number of Loads the MobileNet module. Produces results as video frames with classification Composes frames with contents from its sink pads. Pushes the GStreamer buffers containing these composed Waylandsink submits the video stream received on its sink Weston renders the video stream on a local display. Receives the video stream on its sink pad. Saves the stream as a H.264-encoded MP4 file. Serves as a network sink. Transmits UDP packets to the network. num-camera: The number of inputs from thecamera: The input camera ifnum-camera=1.input-file-path: The directory path to theinput-rtsp-path: The address of the RTSPrtsp://<ip>:<port>/<stream> output-file-path: The directory path tooutput-ip-address: The IP address of theoutput-port-number: The port number of theoutput-display: The connected display