# Pose detection
The **gst-ai-pose-detection** application allows you to detect the body pose of the subject in an image or video. The use case processes input streams from a camera, file, or an RTSP source and uses LiteRT and Qualcomm AI Engine direct models for pose detection. The results are either displayed on the screen, saved as an encoded MP4 file, or streamed over the RTSP server.
The following figure shows the pipeline, which receives the input from a live camera feed, file, USB source, or an RTSP stream, preprocesses it, conducts inference on AI hardware, and generates the output.
This process allows for real-time pose detection and visualization of human poses.
For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-pose-detection.html#section-pqq-1ny-kbc).
**Figure : gst-ai-pose-detection pipeline**
## Input and output capabilities
The following table summarizes the input and output capabilities supported by the sample application:
| Config | Input | Input | Input | Input | Output | Output | Output |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Config | File src | RTSP | USB camera | MIPI camera | File | Display | RTSP |
| Config #1 | Yes | Yes | No | No | No | Yes | Yes |
| Config #2 | Yes | Yes | No | Yes | Yes | Yes | Yes |
| | | | | | | | |
## Sample model and label files
Table : Sample model and label files for gst-ai-pose-detection
| Runtime | Model files | Label files |
| --- | --- | --- |
| LiteRT | hrnet\_pose\_quantized.tflite |
hrnet_pose.json
hrnet_settings.json
|
## Run the application on the target device
The sample application uses the `/etc/configs/config_pose.json` file to read the input parameters.
To create your own config JSON file, use [config_pose.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-pose-detection/config_pose.json?ref_type=heads) as a reference.
1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-80021-50/topic/download-model-and-label-files.html).
2. Update the config JSON file based on the model, input stream, and other properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-pose-detection.html#section-wpt-ysq-32c).
3. Use the following format of the `config_pose.json` file:
{
"file-path": "",
"ml-framework": "",
"model": "",
"labels": "",
"pose-settings-path": "",
"output-type": "waylandsink or filesink or rtspsink",
"runtime": ""
}
Copy to clipboard
Note
To change the threshold, you must configure the confidence value in the `hrnet_settings.json` file.
An example format file for Config #2 and Config #1 is as follows:
Tab Config #2
Tab Config #1
For example, run the application using LiteRT, input video file, custom model, custom label file, DSP runtime:
{
"file-path": "/etc/media/video.mp4",
"ml-framework": "tflite",
"model": "/etc/models/hrnet_pose_quantized.tflite",
"labels": "/etc/labels/hrnet_pose.json",
"pose-settings-path":"/etc/labels/hrnet_settings.json",
"runtime": "dsp",
"output-type": "waylandsink"
}
Copy to clipboard
For example, run the application using LiteRT, input video file, custom model, custom label file, CPU runtime:
{
"file-path": "/etc/media/video.mp4",
"ml-framework": "tflite",
"model": "/etc/models/hrnet_pose_quantized.tflite",
"labels": "/etc/labels/hrnet_pose.json",
"pose-settings-path":"/etc/labels/hrnet_settings.json",
"runtime": "cpu",
"output-type": "waylandsink"
}
Copy to clipboard
Note
Config #1 supports only LiteRT models and the CPU runtime.
4. Run the gst-ai-pose-detection application:
gst-ai-pose-detection --config-file=/etc/configs/config_pose.json
Copy to clipboard
Note
For USB camera input, set the `video-format`, `resolution`, and `framerate` parameters in the configuration file to match the capabilities of the camera. To check the camera capabilities, see [Configure USB camera](https://docs.qualcomm.com/bundle/publicresource/topics/80-80021-8/usb.html#configure-usb-camera).
5. To display the available help options, run the following command in the SSH shell:
gst-ai-pose-detection -h
Copy to clipboard
6. To stop the use case, use **CTRL + C**.
## Expected output
The displayed output shows the detected pose of the objects.

**Figure : Expected output for gst-ai-pose-detection application**
## Pipeline flow
The following table lists the plugins used in the pose detection pipeline:
| Plugin | Description |
| --- | --- |
| Camera source: [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-80021-50/topic/qtiqmmfsrc.html) |
Captures the live stream from camera.
Uses tee to split the stream for inferencing.
|
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.
Uses tee to split the stream for inferencing.
|
| USB camera source: v4l2src |
Captures the live stream from USB camera.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlvconverter.html) |
Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
Converts the preprocessed video stream to a tensor stream on its source pad.
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimltflite](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimltflite.html) and [qtimlqnn](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlqnn.html) | Uses the HRNet model for pose detection.
The application runs on the external delegate to run the model using the Hexagon Tensor Processor.
After the inference runtime receives the tensor stream on its sink pad, qtimltflite does the following:
Runs the inference.
Produces a tensor stream containing the inference results, which is then made available on its source pad.
Manages the inference results from any pose detection model.
Forwards the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
| filesink | Receives the video stream on sink pad and saves it as an H.264-encoded MP4 file. |
| qtirtspbin |
Serves as a network sink.
Transmits UDP packets to the network.
|
## Config JSON field description
The different parameters available to configure the JSON file and run the use case are as follows:
Table : Field description–config_pose.json file
| Field | Values/description |
| --- | --- |
| **ml-framework** | Enable and use one of the following models:
tflite: LiteRT
qnn: Qualcomm AI Engine direct
|
| **runtime** | Enable and use one of the following runtimes:
cpu
gpu
dsp
|
| **Input source** | Enable and use one of the following input sources:
camera: Primary (0) or secondary (1).
file-path: The directory path to the video file.
rtsp-ip-port: The address of the RTSP stream: rtsp://<ip>:<port>/<stream>.
enable-usb-camera: Set to TRUE or FALSE.
|
| **output-ip-address** | Output server IP address. |
| **port** | Output server port. |
| **output-type** | Use one of the following output-type:
waylandsink: To display output on Wayland.
filesink: To store output in file.
rtspsink: To stream output on server.
|
| **USB camera video-format and resolution** | Use one of the following video-format:
nv12
yuy2
mjpeg
Use one of the following resolution fields:
width: Input USB camera source resolution width.
height: Input USB camera source resolution height.
framerate: Input USB camera source framerate.
|
| **enable-usb-camera** | Set to TRUE or FALSE. |
| **output-file** | Output filename. The default output file is output\_pose.mp4. |
## Known issues
Tab Config #2
Tab Config #1
- Lag is observed in pose detection with camera source and file source as quantized models aren't supported in the TFlite IM SDK framework.
- The application may intermittently hang during the `gst_deinit` phase.
- The GPU delegate doesn't function in the QNN and TFLite IM SDK frameworks.
- Detection accuracy may decrease when objects are far from the camera.
- The application identifies the pose of only one person, even when multiple people are present in the frame.
- USB camera input isn't supported.
- Video encoding use case isn't supported.
Note
For better accuracy and detection results, use the **gst-ai-daisychain-detection-pose** application.
## Related information
- [Pose estimation and display with LiteRT](https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-pose-estimation-and-display.html)
- [Pose estimation and encode with LiteRT](https://docs.qualcomm.com/doc/80-80021-50/topic/single-camera-stream-with-pose-estimation-and-encode.html)
Last Published: Mar 26, 2026
[Previous Topic
Object detection](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/gst-ai-object-detection.md) [Next Topic
Image segmentation](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/gst-ai-segmentation.md)
Source: [https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-pose-detection.html](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-pose-detection.html)