# Parallel AI fusion
Source: [https://docs.qualcomm.com/doc/80-70015-50/topic/gst-ai-parallel-inference.html](https://docs.qualcomm.com/doc/80-70015-50/topic/gst-ai-parallel-inference.html)
The **gst-ai-parallel-inference** application enables you to perform object
detection, object classification, pose detection, and image segmentation on an input stream
from different sources such as a camera, a file, or an RTSP network. The use cases use the
Qualcomm Neural Processing SDK runtime for object detection and image segmentation, and
TFLite runtime for classification and pose detection.
Note: This application is not supported on QCS9075.
The figure shows the pipeline, which takes the input from a camera, file, or an RTSP
stream, performs the parallel inferencing for the four use cases, and display the
results side by side on the screen.
For information on the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70015-50/topic/gst-ai-parallel-inference.html#gst-ai-parallel-inference__section_gcg_r3s_lbc).
Figure : gst-ai-parallel-inference pipeline
Learn how to enhance your development projects by performing the four parallel AI
inferences on a live camera stream. This video shows the step-by-step procedure on how
to set up the pipeline and display the results side by side.
## Prerequisites
- To run the application, push the model and label files to the device. For
information on downloading the models, see the following:
- [Download model and label files for Qualcomm Neural Processing SDK](https://docs.qualcomm.com/doc/80-70015-50/topic/ai-ml-sample-applications.html#ai-ml-sample-applications__section_chl_dgz_scc)
- [Download model and label files for TFLite from AI Hub](https://docs.qualcomm.com/doc/80-70015-50/topic/ai-ml-sample-applications.html#ai-ml-sample-applications__section_fsl_lgz_scc)
The application supports both the Qualcomm Neural Processing SDK and
TFLite models.
- To access your host device, enable SSH. For instructions, see [Use SSH](https://docs.qualcomm.com/bundle/publicresource/topics/80-70015-254/how_to.html#use-ssh).
- Enter the SSH shell and run the use cases:
ssh root@Copy to clipboard
- Enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
- Push the files from the host
machine:
scp root@:/opt/Copy to clipboard
- Push the video.mp4 file to the opt
folder.
## Use cases
- Run the
application:
gst-ai-parallel-inference [OPTION?]Copy to clipboard
- Display the help
options:
gst-ai-parallel-inference -hCopy to clipboard
The table lists the model and the inferencing runtime application for each use
case:
| Use case | Inferencing runtime | Model |
| --- | --- | --- |
| [Object detection](https://docs.qualcomm.com/doc/80-70015-50/topic/gst-ai-object-detection.html) | Qualcomm Neural Processing SDK inferencing | [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) |
| [Image segmentation](https://docs.qualcomm.com/doc/80-70015-50/topic/gst-ai-segmentation.html) | Qualcomm Neural Processing SDK inferencing | [DeepLab v3](https://pytorch.org/vision/stable/models/generated/torchvision.models.segmentation.deeplabv3_resnet50.html) |
| [Classification](https://docs.qualcomm.com/doc/80-70015-50/topic/gst-ai-classification.html) | TFLite inferencing | [Inception v3](https://aihub.qualcomm.com/iot/models/inception_v3_quantized?domain=Computer+Vision&useCase=Image+Classification&chipsets=QCS6490%22) |
| [Pose detection](https://docs.qualcomm.com/doc/80-70015-50/topic/gst-ai-pose-detection.html) | TFLite inferencing | [HRNetPose](https://aihub.qualcomm.com/iot/models/hrnet_pose_quantized?searchTerm=hrnet) |
| | | |
Table : Parallel inferencing commands
| Source | Command |
| :--- | :--- |
| Camera | gst-ai-parallel-inference --camera=0Copy to clipboard |
| File | gst-ai-parallel-inference --file-path="/opt/video.mp4"Copy to clipboard |
| RTSP stream | gst-ai-parallel-inference --rtsp-ip-port="rtsp://:/"Copy to clipboard |
To stop the use case, press CTRL + C.
## Expected output
After performing the four parallel inferences, the results are displayed side by side
on the screen.
Figure : Expected output for gst-ai-parallel-inference application
## Pipeline flow
The table lists the plugins used in the parallel inference pipeline:
| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70015-50/topic/qtiqmmfsrc.html) |
Captures the live stream from camera.
Uses tee to split the stream for inferencing.
|
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70015-50/topic/v4l2h264dec.html) | Decodes the video |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-70015-50/topic/qtimlvconverter.html) |
Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimlsnpe](https://docs.qualcomm.com/doc/80-70015-50/topic/qtimlsnpe.html) and [qtimltflite](https://docs.qualcomm.com/doc/80-70015-50/topic/qtimltflite.html) | **qtimlsnpe**: Acts as the ML inferencing plugin, which is used with the Qualcomm Neural Processing SDK runtime. It uses YOLO-NAS for object detection and DeepLab v3 for image segmentation.
**qtimltflite**: Acts as an alternative option to qtimlsnpe. It runs on the TFLite runtime and uses the PoseNet model for pose detection and Inception v3 for classification.
After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
|
| Postprocessing plugins | Handles the inference results from any object detection, classification, pose detection, and segmentation model.
Applies the threshold to the chosen number of results.
Loads the corresponding modules for various pose estimation models. For the use cases described in this section, qtimlvpose loads the PoseNet module.
Produces results as video frames with poses drawn, sending them to the sink pad of the qtivcomposer.
qtimlvsegmentation: Converts the inference tensors that it receives on its sink pad into video formats that the multimedia plugins for further processing.
Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
## Known issues
Video streaming using the RTSP sink encounters a hang issue after running the use
case for a few minutes.
**Parent Topic:** [AI/ML sample applications](https://docs.qualcomm.com/doc/80-70015-50/topic/ai-ml-sample-applications.html)
Last Published: Oct 27, 2025
[Previous Topic
Image segmentation](https://docs.qualcomm.com/bundle/publicresource/80-70015-50/topics/gst-ai-segmentation.md) [Next Topic
Multi-input AI inferencing](https://docs.qualcomm.com/bundle/publicresource/80-70015-50/topics/gst-ai-multi-input-output-object-detection.md)