# Face recognition
The **gst-ai-face-recognition** application collects the live video input from a camera or an RTSP stream and shares this input for face detection, facial landmarking, and face recognition. It uses the `face_det_quantized` models for face detection, `facemap_3dmm_quantized` model for facial landmarking, and `face_attrib_net_quantized` model for face recognition labels.
The result is a preview of the overlaid AI model on the HDMI display.
Note
This application isn't supported in the QLI 2.0 RC2 release.
The following figure shows the pipeline, which receives the input, preprocesses it, runs inferences on AI hardware, and displays the results on the screen.
**Figure : gst-ai-face-recognition pipeline**
For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-face-recognition.html#section-mrl-x4m-qdc).
## Sample model and label files
Table : Sample model and label files for gst-ai-face-recognition
| Runtime | Model files | Label files |
| --- | --- | --- |
| Qualcomm AI Engine direct and LiteRT |
face_det_lite_quantized.tflite
facemap_3dmm_quantized.tflite
face_attrib_net_quantized.tflite
face_det_lite_quantized.bin
facemap_3dmm_quantized.bin
face_attrib_net_quantized.bin
|
face_detection.json
face_recognition_settings.json
face_recognition.json
facemap_3dmm_settings.json
|
## Register a face for facial recognition
Before running the gst-ai-face-recognition application, you can register a face for secure verification and authentication.
1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-80021-50/topic/download-model-and-label-files.html).
2. To register a face, use the following gst-pipeline on the target device shell:
gst-pipeline-app -e \
qtimlvconverter name=stage_01_preproc mode=image-batch-non-cumulative \
qtimltflite name=stage_01_inference model=/etc/models/face_det_lite_quantized.tflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so \
external-delegate-options="QNNExternalDelegate,backend_type=htp;" \
qtimlpostprocess name=stage_01_postproc settings="{\"confidence\": 40.0}" results=4 module=qfd labels=/etc/labels/face_detection.json \
qtimlvconverter name=stage_03_preproc mode=roi-batch-cumulative \
qtimltflite name=stage_03_inference model=/etc/models/face_attrib_net_quantized.tflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so \
external-delegate-options="QNNExternalDelegate,backend_type=htp;" \
qtiqmmfsrc video_0::type=video name=camsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! waylandsink fullscreen=true sync=false \
camsrc.image_1 ! video/x-raw,width=1920,height=1080 ! qtivtransform ! video/x-raw,format=NV12 ! tee name=t_split_1 \
t_split_1. ! queue ! metamux_1. \
t_split_1. ! queue ! stage_01_preproc. stage_01_preproc. ! queue ! stage_01_inference. stage_01_inference. ! queue ! \
stage_01_postproc. stage_01_postproc. ! text/x-raw ! queue ! metamux_1. \
qtimetamux name=metamux_1 ! queue ! tee name=t_split_3 \
t_split_3. ! queue ! stage_03_preproc. stage_03_preproc. ! queue ! stage_03_inference. stage_03_inference. ! queue ! \
multifilesink location=/etc/data/tensor_%d.bin sync=true async=false enable-last-sample=false
Copy to clipboard
A list of options is displayed.
3. To prepare for capturing a facial image, do the following:
1. Select the following options from the list:
1. `(3)PLAYING`: Move the pipeline to the Playing state.
2. `(p)Plugin Mode` ➔ `(26)camsrc`➔ `(70)capture-image`: Capture the image using a camera source.
2. Using the live preview on the display, face the camera and ensure that the camera is pointed straight and there is only one person in the frame.
3. In the terminal, enter 1 for the following values:
1. `GstImageCaptureMode` for `arg0`.
2. `guint` for `arg1`.
4. To capture all the sides of your face, select `capture-image` do the following for each side:
1. Left and right: Turn your head left by 40° while keeping the landmarks visible, then repeat steps 3 and 4. Turn your head right (by 40°) and repeat.
2. Up and down: Raise your head by 30° while keeping the landmarks visible, then repeat steps 3 and 4. Lower your head (by 30°) and repeat.
5. To stop the pipeline, use `(b)Back` and `(q)Quit`.
After running the pipeline, five individual tensor bins are created (`tensor_0.bin` to `tensor_4.bin`) with facial properties recorded for each side of the face.
6. On the target device, go to `/etc/data/`, find the tensor bins. To pull the bins from the target device to the Linux host computer:
- For Qualcomm Linux, run the following commands:
scp root@:/etc/data/tensor_0.bin .
Copy to clipboard
scp root@:/etc/data/tensor_1.bin .
Copy to clipboard
scp root@:/etc/data/tensor_2.bin .
Copy to clipboard
scp root@:/etc/data/tensor_3.bin .
Copy to clipboard
scp root@:/etc/data/tensor_4.bin .
Copy to clipboard
- For Ubuntu Server use the following reference command:
scp ubuntu@:/etc/data/tensor_.bin
Copy to clipboard
7. To merge the tensor bins with all the facial properties into a cohesive image, download and run the `facedb.py` script in the same directory as the tensor bins on the Linux host computer.
1. Download the `facedb.py` script:
curl -L -O https://raw.githubusercontent.com/quic/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/facedb.py
Copy to clipboard
2. Run the script. Note that *<Name of the person>* is case and style sensitive. Ensure that you use the same name consistently.
python3 ./facedb.py "" 512 32 tensor_0.bin tensor_1.bin tensor_2.bin tensor_3.bin tensor_4.bin
Copy to clipboard
A `face.bin` binary is created.
8. Push the `face.bin` binary to `/etc/data` directory and rename it to `face0.bin`.
scp face.bin root@:/etc/data/face0.bin
Copy to clipboard
For Ubuntu Server, copy the `face.bin` binary to the user home directory and then use the `sudo` command to copy it to the `/etc/models` directory.
9. To generate the `face_recognition.json` file and register the new person into the database, use the following reference label file for two-person registered face:
[
{"id": 0, "color": "0x00FF00FF", "label": ""},
{"id": 1, "color": "0xFFFF00FF", "label": ""}
]
Copy to clipboard
Note
Update the ID field according to the number in the list. If more faces are registered, add the structure in a new line within `face_recognition.json`.
10. To generate the `face_recognition_settings.json` file use the following reference label file:
{
"confidence": 51.0,
"databases":[
{"id": 0, "database": "/etc/data/face0.bin"},
{"id": 1, "database": "/etc/data/face1.bin"}
]
}
Copy to clipboard
11. To push the updated `face_recognition.json` and `face_recognition_settings.json` files to the `/etc/labels` directory on the target device.
scp face_recognition.json root@:/etc/labels
Copy to clipboard
scp face_recognition_settings.json root@:/etc/labels
Copy to clipboard
For Ubuntu Server, copy the `face_recognition.json` and `face_recognition_settings.json` files to the user `home` folder and then use the `sudo` command to copy it to the `/etc/labels` directory.
## Run the application on the target device
Note
The following commands provide the default model and label paths. If you have a different folder structure, replace the default paths in the command-line parameters. See [Sample model and label files](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-face-recognition.html#section-bxr-x4m-qdc).
The sample application uses the `/etc/configs/config-face-recognition.json` file to read the input parameters.
To create your own config JSON file, use [config-face-recognition.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/tree/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-face-recognition?ref_type=heads) as a reference.
1. Update the config JSON file based on the model, input stream, and other properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-object-detection.html#section-qjx-hqq-32c).
2. Use the following format of the `/etc/configs/config-face-recognition.json` file:
{
"ml-framework": "",
"face-detection-model": "",
"face-landmark-model":"",
“face-recognition-model”:””,
"face-detection-labels":””,
"face-recognition-labels":””,
"face-recognition-settings": "",
"facemap-3dmm-settings": ""
}
Copy to clipboard
For example, run the application using LiteRT, custom models, custom labels file:
{
"ml-framework":"tflite",
"face-detection-model":"/etc/models/face_det_lite_quantized.tflite",
"face-landmark-model":"/etc/models/facemap_3dmm_quantized.tflite",
"face-recognition-model":"/etc/models/face_attrib_net_quantized.tflite",
"face-detection-labels": "/etc/labels/face_detection.json",
"face-recognition-labels": "/etc/labels/face_recognition.json",
"face-recognition-settings": "/etc/labels/face_recognition_settings.json",
"facemap-3dmm-settings": "/etc/labels/facemap_3dmm_settings.json"
}
Copy to clipboard
3. Run the gst-ai-face-recognition application:
gst-ai-face-recognition --config-file=/etc/configs/config-face-recognition.json
Copy to clipboard
4. To display the available help options, run the following commands in the SSH shell:
gst-ai-face-recognition -h
Copy to clipboard
5. To stop the use case, use **CTRL + C**.
## Expected output

## Pipeline flow
The following table lists the plugins used in the face recognition pipeline:
| Plugin | Description |
| --- | --- |
| Camera source: [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-80021-50/topic/qtiqmmfsrc.html) |
Captures the live stream from camera.
Uses tee to split the stream for inferencing.
|
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlvconverter.html) |
Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
Converts the preprocessed video stream to a tensor stream on its source pad.
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimlqnn](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlqnn.html) | Acts as the inferencing plugin for Qualcomm Neural Network model.
After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
Receives the output of the face detection models from qtimlpostprocess and multiplexes it.
Receives the output of facial pose from qtimlpostprocess and multiplexes it.
|
| tee | Splits the stream for inferencing. |
| qtimlpostprocess for pose estimation | Uses lite-3dmm module to perform the facial pose recognition. |
| qtimlpostprocess for classification model | Uses qfr module to receive the stream from qtimetamux and classifies the face. |
| [qtivoverlay](https://docs.qualcomm.com/doc/80-80021-50/topic/qtioverlay.html) |
Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
## Config JSON field description
Table : Field description–config-face-recognition.json file
| Field | Values/description |
| --- | --- |
| **ml-framework** | Use one of the following models:
tflite–LiteRT
qnn–Qualcomm AI Engine direct
|
| **Models and labels** | See [Sample model and label files](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-face-recognition.html#section-bxr-x4m-qdc).
face-detection-model: The path to the face detection model.
face-landmark-model: The path to the face landmark model.
face-recognition-model: The path to the face recognition model.
face-detection-labels: The path to the face detection labels.
face-recognition-labels: The path to the face recognition labels.
face-recognition-settings: The path of face recognition setting labels.
facemap-3dmm-settings: The path of facemap-3dmm setting labels.
|
## Related information
- [Object detection](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-object-detection.html)
- [Pose detection](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-pose-detection.html)
Last Published: Mar 26, 2026
[Previous Topic
Face detection](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/gst-ai-face-detection.md) [Next Topic
Audio classification](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/audio-classification.md)