# Face recognition
Source: [https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-face-recognition.html](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-face-recognition.html)
The **gst-ai-face-recognition** application collects the live video input from a
camera or an RTSP stream and shares this input for face detection, facial landmarking, and
face recognition. It uses the face\_det\_quantized models for face
detection, `facemap_3dmm_quantized` model for facial landmarking, and
`face_attrib_net_quantized` model for face recognition
labels.
The result is a preview of the overlaid AI model on the HDMI display.
Note: This application isn't supported on the Ubuntu Server.
The following figure shows the pipeline, which receives the input, preprocesses it, runs
inferences on AI hardware, and displays the results on the screen.
Figure : gst-ai-face-recognition pipeline
For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_mrl_x4m_qdc).
## Sample model and label files
Table : Sample model and label files for gst-ai-face-detection
| Runtime | Model files | Label files |
| :--- | :--- | :--- |
| Qualcomm AI Engine direct and LiteRT |
face_det_lite_quantized.tflite
facemap_3dmm_quantized.tflite
face_attrib_net_quantized.tflite
face_det_lite_quantized.bin
facemap_3dmm_quantized.bin
face_attrib_net_quantized.bin
|
face_detection.json
face_recognition_settings.json
face_recognition.json
facemap_3dmm_settings.json
|
- Download the following LiteRT models of w8a8 precision. The model file names in
the following path is subject to change. Ensure to update the model name when
running the commands.
- [face_det_quantized](https://aihub.qualcomm.com/iot/models/face_det_lite?searchTerm=fac)
- [face_attrib_net_quantized](https://aihub.qualcomm.com/iot/models/face_attrib_net?searchTerm=fac)
- Push these models to the /etc/models directory on the
target
device.
scp face_det_lite-lightweight-face-detection.tflite root@:/etc/modelsCopy to clipboard
scp face_attrib_net-facial-attribute.tflite root@:/etc/modelsCopy to clipboard
Note: For Ubuntu Server, copy the model file to the user
home folder and then use the `sudo` command to copy the model
files to the `/etc/models` directory.
.
## Register a face for facial recognition
Before running the gst-ai-face-recognition application, you can register a face for
secure verification and authentication.
1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-70022-50/topic/download-model-and-label-files.html).
2. To register a face, use the following gst-pipeline on the target device shell:
gst-pipeline-app -e \
qtimlvconverter name=stage_01_preproc mode=image-batch-non-cumulative \
qtimltflite name=stage_01_inference model=/etc/models/face_det_lite-lightweight-face-detection.tflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so \
external-delegate-options="QNNExternalDelegate,backend_type=htp;" \
qtimlpostprocess name=stage_01_postproc settings="{\"confidence\": 40.0}" results=4 module=qfd labels=/etc/labels/face_detection.json \
qtimlvconverter name=stage_03_preproc mode=roi-batch-cumulative \
qtimltflite name=stage_03_inference model=/etc/models/face_attrib_net-facial-attribute.tflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so \
external-delegate-options="QNNExternalDelegate,backend_type=htp;" \
qtiqmmfsrc video_0::type=video name=camsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! waylandsink fullscreen=true sync=false \
camsrc.image_1 ! video/x-raw,width=1920,height=1080 ! qtivtransform ! video/x-raw,format=NV12 ! tee name=t_split_1 \
t_split_1. ! queue ! metamux_1. \
t_split_1. ! queue ! stage_01_preproc. stage_01_preproc. ! queue ! stage_01_inference. stage_01_inference. ! queue ! \
stage_01_postproc. stage_01_postproc. ! text/x-raw ! queue ! metamux_1. \
qtimetamux name=metamux_1 ! queue ! tee name=t_split_3 \
t_split_3. ! queue ! stage_03_preproc. stage_03_preproc. ! queue ! stage_03_inference. stage_03_inference. ! queue ! \
multifilesink location=/etc/data/tensor_%d.bin sync=true async=false enable-last-sample=falseCopy to clipboard
A
list of options is displayed.
3. To prepare for capturing a facial image, do the following:
1. Select the following options from the list:
1. `(3)PLAYING`: Move the pipeline to the Playing
state.
2. `(p)Plugin Mode` ➔ `(26)camsrc`➔
`(70)capture-image`: Capture the image using
a camera source.
2. Using the live preview on the display, face the camera and ensure that
the camera is pointed straight and there is only one person in the
frame.
3. In the terminal, enter 1 for the following values:
1. `GstImageCaptureMode` for
`arg0`.
2. `guint` for `arg1`.
4. To capture all the sides of your face, select `capture-image`do
the following for each side:
1. Left and right: Turn your head left by 40° while keeping the landmarks
visible, then repeat steps 3 and 4. Turn your head right (by 40°) and
repeat.
2. Up and down: Raise your head by 30° while keeping the landmarks visible,
then repeat steps 3 and 4. Lower your head (by 30°) and repeat.
5. To stop the pipeline, use `(b)Back` and
`(q)Quit`.
After running the pipeline, five individual
tensor bins are created (tensor\_0.bin to
tensor\_4.bin) with facial properties recorded for
each side of the face.
6. On the target device, go to /etc/data/, find the tensor
bins. To pull the bins from the target device to the Linux host computer:
- For Qualcomm Linux, run the following
commands:
scp root@:/etc/data/tensor_0.bin .Copy to clipboard
scp root@:/etc/data/tensor_1.bin .Copy to clipboard
scp root@:/etc/data/tensor_2.bin .Copy to clipboard
scp root@:/etc/data/tensor_3.bin .Copy to clipboard
scp root@:/etc/data/tensor_4.bin .Copy to clipboard
- For Ubuntu Server use the following reference
command:
scp ubuntu@:/etc/data/tensor_.binCopy to clipboard
7. To merge the tensor bins with all the facial properties into a cohesive image,
download and run the facedb.pyscript in the same directory
as the tensor bins on the Linux host computer.
1. Download the facedb.py
script:
curl -L -O https://raw.githubusercontent.com/quic/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/facedb.pyCopy to clipboard
2. Run the script. Note that <Name of the person> is
case and style sensitive. Ensure that you use the same name
consistently.
python3 ./facedb.py "" 512 32 tensor_0.bin tensor_1.bin tensor_2.bin tensor_3.bin tensor_4.binCopy to clipboard
A face.bin binary is created.
8. Push the face.bin binary to
/etc/data directory and rename it to
face0.bin.
scp face.bin root@:/etc/data/face0.binCopy to clipboard
For Ubuntu Server, copy the `face.bin` binary to the user home
directory and then use the `sudo` command to copy it to the
`/etc/models` directory.
9. To generate the face\_recognition.json file and register the
new person into the database, use the following reference label file for
two-person registered face:
[
{"id": 0, "color": "0x00FF00FF", "label": ""},
{"id": 1, "color": "0xFFFF00FF", "label": ""}
]Copy to clipboard
Note: Update the ID field according to the
number in the list. If more faces are registered, add the structure in a new
line within face\_recognition.json.
10. To generate the face\_recognition\_settings.json file use the
following reference label file:
{
"confidence": 51.0,
"databases":[
{"id": 0, "database": "/etc/data/face0.bin"},
{"id": 1, "database": "/etc/data/face1.bin"}
]
}Copy to clipboard
11. To push the updated face\_recognition.json and
face\_recognition\_settings.json files to the
/etc/labels directory on the target
device.
scp face_recognition.json root@:/etc/labelsCopy to clipboard
scp face_recognition_settings.json root@:/etc/labelsCopy to clipboard
For
Ubuntu Server, copy the `face_recognition.json` and
face\_recognition\_settings.json files to the user
`home` folder and then use the `sudo`
command to copy it to the `/etc/labels` directory.
## Run the application on the target device
Note: The following commands provide the default model and label
paths. If you have a different folder structure, replace the default paths in the
command-line parameters. See [Sample model and label files](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_bxr_x4m_qdc).
The sample application uses the
`/etc/configs/config-face-recognition.json` file to read the
input parameters.
To create your own config JSON file, use [config-face-recognition.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/tree/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-face-recognition?ref_type=heads)as a
reference.
1. Update the config JSON file based on the model, input stream, and other
properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_qjx_hqq_32c).
2. Use the following format of the
`/etc/configs/config-face-recognition.json`
file:
{
"ml-framework": "",
"face-detection-model": "",
"face-landmark-model":"",
“face-recognition-model”:””,
"face-detection-labels":””,
"face-recognition-labels":””,
"face-recognition-settings": "",
"facemap-3dmm-settings": ""
}Copy to clipboard
For example, run the application using LiteRT, custom
models, custom labels
file:
{
"ml-framework":"tflite",
"face-detection-model":"/etc/models/face_det_lite_quantized.tflite",
"face-landmark-model":"/etc/models/facemap_3dmm_quantized.tflite",
"face-recognition-model":"/etc/models/face_attrib_net_quantized.tflite",
"face-detection-labels": "/etc/labels/face_detection.json",
"face-recognition-labels": "/etc/labels/face_recognition.json",
"face-recognition-settings": "/etc/labels/face_recognition_settings.json",
"facemap-3dmm-settings": "/etc/labels/facemap_3dmm_settings.json"
}Copy to clipboard
3. Run the gst-ai-face-recognition
application:
gst-ai-face-recognition --config-file=/etc/configs/config-face-recognition.jsonCopy to clipboard
4. To display the available help options, run the following commands in the SSH
shell:
gst-ai-face-recognition -hCopy to clipboard
5. To stop the use case, use CTRL +
C.
## Expected output
## Pipeline flow
The following table lists the plugins used in the face recognition
pipeline:| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70022-50/topic/qtiqmmfsrc.html) |
Captures the live stream from camera.
Uses tee to split the stream for inferencing.
|
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70022-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-70022-50/topic/qtimlvconverter.html) |
Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
Converts the preprocessed video stream to a tensor stream on its source pad.
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimlqnn](https://docs.qualcomm.com/doc/80-70022-50/topic/qtimlqnn.html) | Acts as the inferencing plugin for Qualcomm Neural Network model.
After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
|
| qtimlpostprocess |
Handles inference results from any face detection model.
Applies a threshold to the chosen number of results.
Receives the output of the face detection models from qtimlpostprocess and multiplexes it.
Receives the output of facial pose from qtimlpostprocess and multiplexes it.
|
| tee | Splits the stream for inferencing. |
| qtimlpostprocess for pose estimation | Uses lite-3dmm module to perform the facial pose recognition. |
| qtimlpostprocess for classification model | Uses qfr module to receive the stream from qtimetamux and classifies the face. |
| [qtivoverlay](https://docs.qualcomm.com/doc/80-70022-50/topic/qtioverlay.html) |
Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
## Config JSON field description
Table : Field description–config-face-recognition.json file
| Field | Values/description |
| :--- | :--- |
| **ml-framework** | Use one of the following models:
tflite–LiteRT
qnn–Qualcomm AI Engine direct
|
| **Models and labels** | See [Sample model and label files](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_bxr_x4m_qdc).
face-detection-model: The path to the face detection model.
face-landmark-model: The path to the face landmark model.
face-recognition-model: The path to the face recognition model.
face-detection-labels: The path to the face detection labels.
face-recognition-labels: The path to the face recognition labels.
face-recognition-settings: The path of face recognition setting labels.
facemap-3dmm-settings: The path of facemap-3dmm setting labels.
|
## Related information
- [Object detection](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-object-detection.html)
- [Pose detection](https://docs.qualcomm.com/doc/80-70022-50/topic/gst-ai-pose-detection.html)
**Parent Topic:** [Run AI/ML sample applications](https://docs.qualcomm.com/doc/80-70022-50/topic/ai-ml-sample-applications.html)
Last Published: Feb 20, 2026
[Previous Topic
Face detection](https://docs.qualcomm.com/bundle/publicresource/80-70022-50/topics/gst-ai-face-detection.md) [Next Topic
Audio classification](https://docs.qualcomm.com/bundle/publicresource/80-70022-50/topics/audio-classification.md)