# Parallel AI fusion
Source: [https://docs.qualcomm.com/doc/80-70014-50/topic/gst-ai-parallel-inference.html](https://docs.qualcomm.com/doc/80-70014-50/topic/gst-ai-parallel-inference.html)
The **gst-ai-parallel-inference** application enables you to perform object
detection, object classification, pose detection, and image segmentation on a live camera
stream. The use cases use Qualcomm Neural Processing SDK runtime for object detection and
image segmentation, and TFLite runtime for classification and pose detection.
The figure shows the pipeline, which performs the parallel inferencing for the four use
cases. For information on the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70014-50/topic/gst-ai-parallel-inference.html#gst-ai-parallel-inference__section_gcg_r3s_lbc).
Figure : gst-ai-parallel-inference pipeline
Learn how to enhance your development projects by performing the four parallel AI
inferences on a live camera stream. This video shows the step-by-step procedure on how
to set up the pipeline and display the results side-by-side.
## Prerequisites
- Push the model and label files to the device to run the application. For more
information, see [Download the model and label files](https://docs.qualcomm.com/doc/80-70014-50/topic/ai-ml-sample-applications.html#ai-ml-sample-applications__prereq_v23_cxc_gbc).
- Enable SSH in Permissive mode to securely access your host device. For
instructions, see [How to SSH?](https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-254/how_to.html#how-to-ssh-)
- Use the following command to enter the SSH shell and execute the use
cases:
ssh root@Copy to clipboard
- Run the following command to enable the Permissive
mode:
setenforce 0Copy to clipboard
- Run the following command to enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
- Run the following command to push the files from host
machine:
scp root@:/opt/Copy to clipboard
## Use cases
Use the following command to execute the application:
gst-ai-parallel-inferenceCopy to clipboard
To display the available help options, run the following
command:
gst-ai-parallel-inference -hCopy to clipboard
To
stop the use case, press CTRL + C.
The table lists the model and the inferencing runtime application for each use case.
Click the respective link to add model and label files.
| Use case | Model |
| --- | --- |
| [Object detection](https://docs.qualcomm.com/doc/80-70014-50/topic/gst-ai-object-detection.html) | Qualcomm Neural Processing SDK inferencing – [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) model |
| [Classification](https://docs.qualcomm.com/doc/80-70014-50/topic/gst-ai-classification.html) | TFLite inferencing – [Inception v3](https://pytorch.org/hub/pytorch_vision_inception_v3/) model |
| [Pose detection](https://docs.qualcomm.com/doc/80-70014-50/topic/gst-ai-pose-detection.html) | TFLite inferencing – [PoseNet MobileNet v1](https://github.com/google-coral/project-posenet/tree/master) model |
| [Image segmentation](https://docs.qualcomm.com/doc/80-70014-50/topic/gst-ai-segmentation.html) | Qualcomm Neural Processing SDK inferencing – [DeepLab v3](https://pytorch.org/vision/stable/models/generated/torchvision.models.segmentation.deeplabv3_resnet50.html) model |
## Expected output
After performing the four parallel inferences, the results are displayed side-by-side
on the screen.
Figure : Expected output for gst-ai-parallel-inference application
## Pipeline flow
The table lists the plugins used to execute the parallel inference
pipeline:| Plugin | Description |
| --- | --- |
| [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70014-50/topic/qtiqmmfsrc.html) | Captures the camera live stream and employs the tee to split the stream for inferencing. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-70014-50/topic/qtimlvconverter.html) |
Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimlsnpe](https://docs.qualcomm.com/doc/80-70014-50/topic/qtimlsnpe.html) and [qtimltflite](https://docs.qualcomm.com/doc/80-70014-50/topic/qtimltflite.html) | **qtimlsnpe**: Acts as the ML inferencing plugin, which is used with the Qualcomm Neural Processing SDK runtime. It uses YOLO-NAS for object detection and DeepLab v3 for image segmentation.
**qtimltflite**: Acts as an alternative option to qtimlsnpe. It runs on the TFLite runtime and uses the PoseNet model for pose detection and Inception v3 for classification.
After the inference runtime receives the tensor stream on its sink pad, it executes the inference.
Produces a tensor stream with the inference results on its source pad.
|
| Postprocessing plugins | Handles the inference results from any object detection, classification, pose detection, and segmentation model.
Applies the threshold to the chosen number of results.
Loads the corresponding modules for various pose estimation models. For the use cases described in this section, qtimlvpose loads the PoseNet module.
Produces results as video frames with poses drawn, sending them to the sink pad of the qtivcomposer.
qtimlvsegmentation: Converts the inference tensors that it receives on its sink pad into video formats that the multimedia plugins for further processing.