# Parallel inferencing
Source: [https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-parallel-inference.html](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-parallel-inference.html)
The **gst-ai-parallel-inference** application allows you to perform object
detection, object classification, pose detection, and image segmentation on an input stream
from different sources such as a camera, a file, or an RTSP network. The use cases use the
LiteRT models for object detection, image segmentation, classification, and pose
detection.
Note: This
application isn't supported on QCS8275.
The figure shows the pipeline, which takes the input from a camera, file, or an RTSP
stream, performs the parallel inferencing for the four use cases, and display the
results side by side on the screen.
For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-parallel-inference.html#gst-ai-parallel-inference__section_gcg_r3s_lbc).
Figure : gst-ai-parallel-inference pipeline
Enhance your development projects by performing the four parallel AI inferences on a live
camera stream. This video shows the step-by-step procedure on how to set up the pipeline
and display the results side by side. The video will open in a new tab.
## Sample model and label files
Table : Sample model and label files for gst-ai-parallel-inference
| Application | Model files | Label files |
| --- | --- | --- |
| Object detection | yolov8_det_quantized.tflite | yolov8.labels |
| Pose estimation | hrnet_pose_quantized.tflite | hrnet_pose.labels |
| Segmentation | deeplabv3_plus_mobilenet_quantized.tflite | deeplabv3_resnet50.labels |
| Classification | inception_v3_quantized.tflite | classification.labels |
## Prerequisites
- If not already done so, [Download and install eSDK](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-51/install-sdk.html#download-and-install-esdk-).
- [Download model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html).
- To access your host computer, enable SSH. For instructions, see [Sign in using SSH](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-254/how_to.html#use-ssh).
Note: If SSH is already enabled, you can skip this
step.
- Push the model files from the Linux host
computer.
scp root@:/etc/modelsCopy to clipboard
- Note that the [downloaded
script](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html) downloads the sample video.mp4 video to
the /etc/media directory. If you are using a custom video,
then ensure that you push the video to /etc/media and
update the file path in the application config.JSON file.
- Connect the display to the device using the HDMI port. For instructions, see
[Set up HDMI display](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/samples.html).
- Enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
If you face issues while enabling camera or display, see [Camera troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-17/troubleshooting.html) and [Display troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/debug.html).
## Run the application
For sample model and label files, see [Sample model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-parallel-inference.html#gst-ai-parallel-inference__section_pnn_hmb_4dc).
Enter SSH shell and copy the YOLO-NAS label files to YOLOv8:
cp /etc/labels/yolonas.labels /etc/labels/yolov8.labelsCopy to clipboard
To run the application, use the following syntax:
gst-ai-parallel-inference [OPTION?]Copy to clipboard
- Run with input from the primary and secondary
cameras:
gst-ai-parallel-inference -c 0Copy to clipboard
gst-ai-parallel-inference -c 1Copy to clipboard
- Run with input from a
file:
gst-ai-parallel-inference -s /etc/media/video.mp4Copy to clipboard
- Run with input from an RTSP
stream:
gst-ai-parallel-inference --rtsp-ip-port="rtsp:///live.mkv"Copy to clipboard
- Run with input from a video stream with constants for the LiteRT models for all
four
inferences:
gst-ai-parallel-inference --file-path="/etc/media/video.mp4" --object-detection-constants="YOLOv8,q-offsets=<21.0, 0.0, 0.0>,q-scales=<3.0546178817749023, 0.003793874057009816, 1.0>;" --pose-detection-constants="Posenet,q-offsets=<8.0>,q-scales=<0.0040499246679246426>;" --segmentation-constants="deeplab,q-offsets=<0.0>,q-scales=<1.0>;" --classification-constants="Inceptionv3,q-offsets=<38.0>,q-scales=<0.17039915919303894>;"Copy to clipboard
To display the available help options, run the following command in the SSH
shell:
gst-ai-parallel-inference -hCopy to clipboard
To stop the use case, use CTRL + C.
## Expected output
After performing the four parallel inferences, the results are displayed side by side
on the screen.
Figure : Expected output for gst-ai-parallel-inference application
## Pipeline flow
The table lists the plugins used in the parallel inference pipeline:| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70018-50/topic/qtiqmmfsrc.html) |
Captures the live stream from camera.
Uses tee to split the stream for inferencing.
|
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70018-50/topic/v4l2h264dec.html) | Decodes the video |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvconverter.html) |
Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimltflite](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimltflite.html) |
After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
|
| Postprocessing plugins | Handles the inference results from any object detection, classification, pose detection, and segmentation model.
Applies the threshold to the chosen number of results.
Loads the corresponding modules for various pose estimation models. For the use cases described in this section, qtimlvpose loads the PoseNet module.
Produces results as video frames with poses drawn, sending them to the sink pad of the qtivcomposer.
qtimlvsegmentation: Converts the inference tensors that it receives on its sink pad into video formats that the multimedia plugins for further processing.