# Face recognition
Source: [https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-face-recognition.html](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-face-recognition.html)
The **gst-ai-face-recognition** application collects the live video input from a
camera or an RTSP stream and shares this input for face detection, facial landmarking and
face recognition. It uses the face\_det\_quantized models for face
detection, `facemap_3dmm_quantized` model for facial landmarking, and
`face_attrib_net_quantized` model for face recognition
labels.
The result is a preview of the overlaid AI model on the HDMI display.
The figure shows the pipeline, which receives the input, performs preprocessing, runs
inferences on AI hardware, and displays the results on the screen.
Figure : gst-ai-face-recognition pipeline
For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_mrl_x4m_qdc).
## Sample model and label files
Table : Sample model and label files for gst-ai-face-detection
| Runtime | Model files | Label files |
| :--- | :--- | :--- |
| Qualcomm AI Engine direct and LiteRT |
- face_det_lite_quantized.tflite
- facemap_3dmm_quantized.tflite
- face_attrib_net_quantized.tflite
- face_det_lite_quantized.bin
- facemap_3dmm_quantized.bin
- face_attrib_net_quantized.bin
|
- face_detection.labels
- face_recognition.labels
- face_landmark.labels
|
## Prerequisites
- If not already done so, [Download and install eSDK](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-51/install-sdk.html#download-and-install-esdk-).
- [Download model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html).
- To access your host computer, enable SSH. For instructions, see [Sign in using SSH](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-254/how_to.html#use-ssh).
Note: If SSH is already enabled, you can skip this
step.
- Push the model files from the Linux host
computer.
scp root@:/etc/modelsCopy to clipboard
- Note that the [downloaded
script](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html) downloads the sample video.mp4 video to
the /etc/media directory. If you are using a custom video,
then ensure that you push the video to /etc/media and
update the file path in the application config.JSON file.
- Connect the display to the device using the HDMI port. For instructions, see
[Set up HDMI display](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/samples.html).
- Enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
If you face issues while enabling camera or display, see [Camera troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-17/troubleshooting.html) and [Display troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/debug.html).
- Download the following Qualcomm AI Engine direct models:
- [face_det_quantized](https://aihub.qualcomm.com/iot/models/face_det_lite_quantized?searchTerm=face)
- [face_attrib_net_quantized](https://aihub.qualcomm.com/iot/models/face_attrib_net_quantized?searchTerm=face)
- Push these models to the /etc/models directory on the
target
device.
scp face_det_lite_quantized.bin root@:/etc/modelsCopy to clipboard
scp face_attrib_net_quantized.bin root@:/etc/modelsCopy to clipboard
## Register a face for facial recognition
Before running the gst-ai-face-recognition application, you can register a face for
secure verification and authentication.
1. To register a face, use the following gst-pipeline on the target device shell:
gst-pipeline-app -e \
qtimlvconverter name=stage_01_preproc mode=image-batch-non-cumulative \
qtimlqnn name=stage_01_inference model=/etc/models/face_det_lite_quantized.bin backend=/usr/lib/libQnnHtp.so \
qtimlvdetection name=stage_01_postproc stabilization=false threshold=40.0 results=4 module=qfd labels=/etc/labels/face_detection.labels \
qtimlvconverter name=stage_03_preproc mode=roi-batch-cumulative \
qtimlqnn name=stage_03_inference model=/etc/models/face_attrib_net_quantized.bin backend=/usr/lib/libQnnHtp.so \
qtiqmmfsrc video_0::type=video name=camsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! waylandsink fullscreen=true sync=false \
camsrc.image_1 ! video/x-raw,width=1920,height=1080 ! qtivtransform ! video/x-raw,format=NV12 ! tee name=t_split_1 \
t_split_1. ! queue ! metamux_1. \
t_split_1. ! queue ! stage_01_preproc. stage_01_preproc. ! queue ! stage_01_inference. stage_01_inference. ! queue ! stage_01_postproc. stage_01_postproc. ! text/x-raw ! queue ! metamux_1. \
qtimetamux name=metamux_1 ! queue ! tee name=t_split_3 \
t_split_3. ! queue ! stage_03_preproc. stage_03_preproc. ! queue ! stage_03_inference. stage_03_inference. ! queue ! multifilesink location=/etc/data/tensor_%d.bin sync=true async=false enable-last-sample=falseCopy to clipboard
A
list of options is displayed.
2. To prepare for capturing a facial image, do the following:
1. Select the following options from the list:
1. `(3)PLAYING`: Move the pipeline to the Playing
state.
2. `(p)Plugin Mode` ➔ `(26)camsrc`➔
`(37)capture-image`: Capture the image using
a camera source.
2. Using the live preview on the display, face the camera and ensure that
the camera is pointed straight and there is only one person in the
frame.
3. In the terminal, enter 1 for the following values:
1. `GstImageCaptureMode` for
`arg0`.
2. `guint` for `arg1`.
3. To capture all the sides of your face, select `capture-image`do
the following for each side:
1. Left and right: Turn your head left by 40° while keeping the landmarks
visible, then repeat steps 3 and 4. Turn your head right (by 40°) and
repeat.
2. Up and down: Raise your head by 30° while keeping the landmarks visible,
then repeat steps 3 and 4. Lower your head (by 30°) and repeat.
4. To stop the pipeline, use `(b)Back` and
`(q)Quit`.
After running the pipeline, five individual
tensor bins are created (tensor\_0.bin to
tensor\_4.bin) with facial properties recorded for
each side of the face.
5. On the target device, go to /etc/data/, find the tensor
bins. To pull the bins from the target device to the Linux host computer, run
the following
commands:
scp root@:/etc/data/tensor_0.bin .Copy to clipboard
scp root@:/etc/data/tensor_1.bin .Copy to clipboard
scp root@:/etc/data/tensor_2.bin .Copy to clipboard
scp root@:/etc/data/tensor_3.bin .Copy to clipboard
scp root@:/etc/data/tensor_4.bin .Copy to clipboard
6. To merge the tensor bins with all the facial properties into a cohesive image,
download and run the facedb.pyscript in the same directory
as the tensor bins on the Linux host computer.
1. Download the facedb.py
script:
curl -L -O https://raw.githubusercontent.com/quic/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/facedb.pyCopy to clipboard
2. Run the script. Note that <Name of the person> is
case and style sensitive. Ensure that you use the same name
consistently.
python3 ./facedb.py "" 512 32 tensor_0.bin tensor_1.bin tensor_2.bin tensor_3.bin tensor_4.binCopy to clipboard
A face.bin binary is created.
7. Push the face.bin binary to
/etc/data directory and rename it to
face0.bin.
scp face.bin root@:/etc/data/face0.binCopy to clipboard
8. To generate the face\_recognition.labels file and register
the new person into the database, use the following reference label file for
two-person registered
face:
(structure)",id=(guint)0x0,database=(string)/etc/data/face0.bin,color=(guint)0x00FF00FF;"Copy to clipboard
(structure)",id=(guint)0x01,database=(string)/etc/data/face1.bin,color=(guint)0x00FF00FF;"Copy to clipboard
Note: Update the ID field according to the number in the
list. If more faces are registered, add the structure in a new line within
face\_recognition.labels.
9. To save the labels, push the updated
face\_recognition.labels file to the
/etc/labels directory on the target
device.
scp face_recognition.labels root@:/etc/labelsCopy to clipboard
## Run the application
Note: The following commands provide the default model and label
paths. If you have a different folder structure, replace the default paths in the
command-line parameters. See [Sample model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_bxr_x4m_qdc).
The sample application uses the
`/etc/configs/config-face-recognition.json` file to read the
input parameters.
To create your own config JSON file, use [config-face-recognition.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/tree/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-face-recognition?ref_type=heads)as a
reference.
1. Use the following format of the
`/etc/configs/config-face-recognition.json`
file:
{
"ml-framework": "",
"face-detection-model": "",
"face-landmark-model":"",
“face-recognition-model”:””,
"face-detection-labels":””,
"face-landmark-labels":””,
"face-recognition-labels":””,
"face-detection-constants":””,
"face-landmark-constants":””,
"face-recognition-constants":””
}Copy to clipboard
Note: Update the config JSON file
based on the model, input stream, and other properties. For more
information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_qjx_hqq_32c).
For
example, run the application using LiteRT, custom models, custom labels
file:
{
"ml-framework":"tflite",
"face-detection-model":"/etc/models/face_det_lite_quantized.tflite",
"face-landmark-model":"/etc/models/facemap_3dmm_quantized.tflite",
"face-recognition-model":"/etc/models/face_attrib_net_quantized.tflite",
"face-detection-labels": "/etc/labels/face_detection.labels",
"face-landmark-labels": "/etc/labels/face_landmark.labels",
"face-recognition-labels": "/etc/labels/face_recognition.labels",
"face-detection-constants": "DET,q-offsets=<178.0, 0.0, 102.0>,q-scales=<0.034
"face-landmark-constants":"DMM,q-offsets=<211.0>,q-scales=<0.06002333015203476
"face-recognition-constants":"qfr,q-offsets=<124.0, 153.0, 125.0, 133.0, 126.0
}Copy to clipboard
2. Run the gst-ai-face-recognition
application:
gst-ai-face-recognition --config-file=/etc/configs/config-face-recognition.jsonCopy to clipboard
To display the available help options, run the following commands in the SSH
shell:
gst-ai-face-recognition -hCopy to clipboard
To stop the use case, use CTRL + C.
## Expected output
## Pipeline flow
The table lists the plugins used in the face recognition pipeline:| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70018-50/topic/qtiqmmfsrc.html) |
- Captures the live stream from camera.
- Uses tee to split the stream for inferencing.
|
| File source: filesrc |
- Captures the video stream using filesrc, followed by
qtdemux, which demultiplexes the stream.
- Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
- Captures the RTSP stream using rtspsrc, followed by
rtph264depay for video extraction.
- Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70018-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvconverter.html) |
- Receives the video stream on its sink pad.
- Performs the following preprocessing on the stream data.
This preprocessing is done when the model expects
floating-point values as input.
- Color conversion
- Scaling (up or down)
- Normalization
- Converts the preprocessed video stream to a tensor stream on
its source pad.
The tensor stream is used for inferencing in the later stages of
the pipeline. |
| [qtimlqnn](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlqnn.html) | Acts as the inferencing plugin for Qualcomm Neural Network
model.
- After the inference runtime receives the tensor stream on
its sink pad, it runs the inference.
- Produces a tensor stream with the inference results on its
source pad.
|
| [qtimlvdetection](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvdetection.html) |
- Handles inference results from any face detection
model.
- Applies a threshold to the chosen number of results.
|
| [qtimetamux](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimetamux.html) |
- Receives the output of the face detection models from
qtimlvdetection and multiplexes it.
- Receives the output of facial pose from qtimlvpose and
multiplexes it.
|
| tee | Splits the stream for inferencing. |
| [qtimlvpose](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvpose.html) | Performs the facial pose recognition. |
| [qtimlvclassification](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlvclassification.html) | Receives the stream from qtimetamux and classifies the
face. |
| [qtivoverlay](https://docs.qualcomm.com/doc/80-70018-50/topic/qtioverlay.html) |
- Receives the multiplexed stream.
- Overlays the bounding boxes on the stream.
|
| [Waylandsink](https://docs.qualcomm.com/doc/80-70018-50/topic/waylandsink.html) |
- Waylandsink submits the video stream received on its sink
pad to Weston.
- Weston renders the video stream on a local display.
|
## Config JSON field description
Table : Field description–config-face-recognition.json file
| Field | Values/description |
| :--- | :--- |
| **ml-framework** | Use one of the following models:
tflite–LiteRT
qnn–Qualcomm AI Engine direct
|
| **Models and labels** | See [Sample model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_bxr_x4m_qdc).
- face-detection-model: The path to the face detection
model.
- face-landmark-model: The path to the face landmark
model.
- face-recognition-model: The path to the face
recognition model.
- face-detection-labels: The path to the face detection
labels.
- face-landmark-labels: The path to the face landmark
labels.
- face-recognition-labels: The path to the face
recognition labels.
|
| **Constants** |
- face-detection-constants: The face detection model
constants.
- face-landmark-constants: The face landmark model
constants.
- face-recognition-constants The face recognition model
constants.
|
## Known issues
The 3D facial landmark isn't observed with LiteRT models.
**Parent Topic:** [Run AI/ML sample applications](https://docs.qualcomm.com/doc/80-70018-50/topic/ai-ml-sample-applications.html)
**Related Resources**
- [Object detection](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-object-detection.html)
- [Pose detection](https://docs.qualcomm.com/doc/80-70018-50/topic/gst-ai-pose-detection.html)
Last Published: Jan 30, 2026
[Previous Topic
Face detection](https://docs.qualcomm.com/bundle/publicresource/80-70018-50/topics/gst-ai-face-detection.md) [Next Topic
Audio classification](https://docs.qualcomm.com/bundle/publicresource/80-70018-50/topics/audio-classification.md)