# Image segmentation
Source: [https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-segmentation.html](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-segmentation.html)
The **gst-ai-segmentation** application allows you to divide an image into
different and meaningful parts or segments and assign a label to each homogenous segment
based on the similarity of the attributes. Use Qualcomm Neural Processing SDK runtime,
Qualcomm AI Engine direct runtime, and LiteRT for image segmentation.
The figure shows the pipeline, which receives the input from a live camera feed, file, or
an RTSP stream, preprocesses the video data, runs inference using AI hardware, and
displays the segmented data on the screen.
For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-segmentation.html#gst-ai-segmentation__section_xb4_p1s_lbc).
Figure : gst-ai-segmentation pipeline
## Sample model and label files
| Runtime | Model files | Label files |
| --- | --- | --- |
| Qualcomm Neural Processing SDK | deeplabv3_resnet50.dlc | deeplabv3_resnet50.labels |
| LiteRT | deeplabv3_plus_mobilenet_quantized.tflite | deeplabv3_resnet50.labels |
| Qualcomm AI Engine direct | deeplabv3_plus_mobilenet_quantized.bin | deeplabv3_resnet50.labels |
| | | |
| | | |
## Prerequisites
- If not already done so, [Download and install eSDK](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-51/install-sdk.html#download-and-install-esdk-).
- To run the application, push the model and label files to the device. For
instructions, see [Download model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html).
The
application supports the Qualcomm Neural Processing SDK, Qualcomm AI Engine
direct, and LiteRT models.
- To access your host computer, enable SSH. For instructions, see [Sign in using SSH](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-254/how_to.html#use-ssh).
Note: If SSH is already enabled, you can skip this
step.
- Push the model files from the Linux host computer:
scp root@:/etc/modelsCopy to clipboard
- Note that the [downloaded
script](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html) downloads the sample video.mp4 video to
the /etc/media directory. If you are using a custom video,
then ensure that you push the video to /etc/media and
update the file path in the application config.JSON file.
- Connect the display to the device using the HDMI port. For instructions, see
[Set up HDMI display](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/samples.html).
- Enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
If you face issues while enabling camera or display, see [Camera troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-17/troubleshooting.html) and [Display troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/debug.html).
## Run the application
The sample application uses the
/etc/configs/config\_segmentation.json file to read the
input parameters.
To create your own config JSON file, use [config_segmentation.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-segmentation/config_segmentation.json?ref_type=heads) as a
reference.
1. Use the following format of the config\_segmentation.json
file:
{
"file-path": "",
"ml-framework": "",
"model": "",
"labels": "",
"constants": "",
"runtime": ""
}Copy to clipboard
Note: Update the config JSON file based
on the model, input stream, and other properties. For more information, see
[Config JSON field description](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-segmentation.html#gst-ai-segmentation__section_ict_rdr_32c).
For
example, run the application using the
DeepLabV3-Plus-MobileNet LiteRT model, DSP runtime,
custom model and label paths, and constants from the video
file:
{
"file-path": "/etc/media/video.mp4",
"ml-framework": "tflite",
"model": "/etc/models/deeplabv3_plus_mobilenet_quantized.tflite",
"labels": "/etc/labels/deeplabv3_resnet50.labels",
"constants": "deeplab,q-offsets=<0.0>,q-scales=<1.0>;",
"runtime": "dsp"
}Copy to clipboard
Note: The values for the
`q-scales` and `q-offsets` constants are
`<1.0>` and `<0.0>`
respectively.
2. Run the gst-ai-segmentation application:
gst-ai-segmentation --config-file=/etc/configs/config_segmentation.jsonCopy to clipboard
To display the available help options, run the following command in the SSH
shell:
gst-ai-segmentation -hCopy to clipboard
To stop the use case, use CTRL + C.
## Expected output
The segmented data is displayed on the local display.
Figure : Expected output for gst-ai-segmentation application
## Pipeline flow
The table lists the plugins used in the image segmentation pipeline:
| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70018-50/topic/qtiqmmfsrc.html) |
- Captures the live stream from camera.
- Uses tee to split the stream for inferencing.
|
| File source: filesrc |
- Captures the video stream using filesrc, followed by
qtdemux, which demultiplexes the stream.
- Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
- Captures the RTSP stream using rtspsrc, followed by
rtph264depay for video extraction.
- Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70018-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvconverter.html) |
- Receives the video stream on its sink pad.
- Performs the following preprocessing on the stream data.
This preprocessing is done when the model expects
floating-point values as input.
- Color conversion
- Scaling (up or down)
- Normalization
- Converts the preprocessed video stream to a tensor stream on
its source pad.
The tensor stream is used for inferencing in the later stages of
the pipeline. |
| Inferencing plugins: |
- After the inference runtime receives the tensor stream on
its sink pad, it runs the inference.
- Produces a tensor stream with the inference results on its
source pad.
|
| [qtimlvsegmentation](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvsegmentation.html) | Converts the inference tensors that it receives on its sink pad
into video formats that the multimedia plugins use for further
processing. |
| [qtivcomposer](https://docs.qualcomm.com/doc/80-70018-50/topic/qtivcomposer.html) |
- Composes frames by combining content from its sink pads.
- Pushes the GStreamer buffers containing these composed
frames to its source pad.
|
| [Waylandsink](https://docs.qualcomm.com/doc/80-70018-50/topic/waylandsink.html) |
- Forwards the video stream received on its sink pad to
Weston.
- Weston renders the video stream on a local display.
|
## Config JSON field description
Table : Field description–config_segmentation.json file
| Field | Values/description |
| :--- | :--- |
| **ml-framework** | Enable and use one of the following models:
snpe: Qualcomm Neural Processing SDK
tflite: LiteRT
qnn: Qualcomm AI Engine direct
|
| **runtime** | Enable and use one of the following runtimes: |
| **Input source** | Enable and use one of the following input sources:
camera: Primary (0) or secondary (1).
file-path: The directory path to the video
file.
rtsp-ip-port: The address of the RTSP
stream:
rtsp://<ip>:<port>/<stream>
|
## Known issues
An FPS drop is observed when using the Qualcomm Neural Processing Engine SDK
models.
**Parent Topic:** [Run AI/ML sample applications](https://docs.qualcomm.com/doc/80-70018-50/topic/ai-ml-sample-applications.html)
**Related Resources**
- [Image segmentation and display with LiteRT](https://docs.qualcomm.com/doc/80-70018-50/topic/single-camera-stream-with-image-segmentation-and-display.html)
- [Image segmentation and encode with LiteRT](https://docs.qualcomm.com/doc/80-70018-50/topic/single-camera-stream-with-image-segmentation-and-encode.html)
- [Image segmentation and display with Neural Processing SDK](https://docs.qualcomm.com/doc/80-70018-50/topic/single-camera-stream-with-image-segmentation-and-display-with-deeplabv3-quantized.html)
- [Image segmentation and encode with Neural Processing SDK](https://docs.qualcomm.com/doc/80-70018-50/topic/single-camera-stream-with-image-segmentation-and-encode-with-deeplabv3-quantized.html)
Last Published: Jan 30, 2026
[Previous Topic
Pose detection](https://docs.qualcomm.com/bundle/publicresource/80-70018-50/topics/gst-ai-pose-detection.md) [Next Topic
Image segmentation using Python with container](https://docs.qualcomm.com/bundle/publicresource/80-70018-50/topics/image-segmentation-using-python.md)