# Face recognition
Source: [https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-face-recognition.html](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-face-recognition.html)
The **gst-ai-face-recognition** application collects the live video input from a
camera or an RTSP stream and shares this input for face detection, facial landmarking, and
face recognition. It uses the face\_det\_quantized models for face
detection, `facemap_3dmm_quantized` model for facial landmarking, and
`face_attrib_net_quantized` model for face recognition
labels.
Note: This application isn't supported on the Ubuntu Server.
The result is a preview of the overlaid AI model on the HDMI display.
The following figure shows the pipeline, which receives the input, preprocesses it, runs
inferences on AI hardware, and displays the results on the screen.
Figure : gst-ai-face-recognition pipeline
For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_mrl_x4m_qdc).
## Sample model and label files
Table : Sample model and label files for gst-ai-face-detection
| Runtime | Model files | Label files |
| :--- | :--- | :--- |
| Qualcomm AI Engine direct and LiteRT |
face_det_lite_quantized.tflite
facemap_3dmm_quantized.tflite
face_attrib_net_quantized.tflite
face_det_lite_quantized.bin
facemap_3dmm_quantized.bin
face_attrib_net_quantized.bin
|
face_detection.labels
face_recognition.labels
face_landmark.labels
|
- Download the following Qualcomm AI Engine direct models:
- [face_det_quantized](https://aihub.qualcomm.com/iot/models/face_det_lite?searchTerm=fac)
- [face_attrib_net_quantized](https://aihub.qualcomm.com/iot/models/face_attrib_net?searchTerm=fac)
- Push these models to the /etc/models directory on the
target
device.
scp face_det_lite_quantized.bin root@:/etc/modelsCopy to clipboard
scp face_attrib_net_quantized.bin root@:/etc/modelsCopy to clipboard
Note: For Ubuntu Server, copy the model file to the user
home folder and then use the `sudo` command to copy the model
files to the `/etc/models` directory.
.
## Register a face for facial recognition
Before running the gst-ai-face-recognition application, you can register a face for
secure verification and authentication.
1. To register a face, use the following gst-pipeline on the target device shell:
gst-pipeline-app -e \
qtimlvconverter name=stage_01_preproc mode=image-batch-non-cumulative \
qtimlqnn name=stage_01_inference model=/etc/models/face_det_lite_quantized.bin backend=/usr/lib/libQnnHtp.so \
qtimlvdetection name=stage_01_postproc stabilization=false threshold=40.0 results=4 module=qfd labels=/etc/labels/face_detection.labels \
qtimlvconverter name=stage_03_preproc mode=roi-batch-cumulative \
qtimlqnn name=stage_03_inference model=/etc/models/face_attrib_net_quantized.bin backend=/usr/lib/libQnnHtp.so \
qtiqmmfsrc video_0::type=video name=camsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! waylandsink fullscreen=true sync=false \
camsrc.image_1 ! video/x-raw,width=1920,height=1080 ! qtivtransform ! video/x-raw,format=NV12 ! tee name=t_split_1 \
t_split_1. ! queue ! metamux_1. \
t_split_1. ! queue ! stage_01_preproc. stage_01_preproc. ! queue ! stage_01_inference. stage_01_inference. ! queue ! stage_01_postproc. stage_01_postproc. ! text/x-raw ! queue ! metamux_1. \
qtimetamux name=metamux_1 ! queue ! tee name=t_split_3 \
t_split_3. ! queue ! stage_03_preproc. stage_03_preproc. ! queue ! stage_03_inference. stage_03_inference. ! queue ! multifilesink location=/etc/data/tensor_%d.bin sync=true async=false enable-last-sample=falseCopy to clipboard
A
list of options is displayed.
2. To prepare for capturing a facial image, do the following:
1. Select the following options from the list:
1. `(3)PLAYING`: Move the pipeline to the Playing
state.
2. `(p)Plugin Mode` ➔ `(26)camsrc`➔
`(70)capture-image`: Capture the image using
a camera source.
2. Using the live preview on the display, face the camera and ensure that
the camera is pointed straight and there is only one person in the
frame.
3. In the terminal, enter 1 for the following values:
1. `GstImageCaptureMode` for
`arg0`.
2. `guint` for `arg1`.
3. To capture all the sides of your face, select `capture-image`do
the following for each side:
1. Left and right: Turn your head left by 40° while keeping the landmarks
visible, then repeat steps 3 and 4. Turn your head right (by 40°) and
repeat.
2. Up and down: Raise your head by 30° while keeping the landmarks visible,
then repeat steps 3 and 4. Lower your head (by 30°) and repeat.
4. To stop the pipeline, use `(b)Back` and
`(q)Quit`.
After running the pipeline, five individual
tensor bins are created (tensor\_0.bin to
tensor\_4.bin) with facial properties recorded for
each side of the face.
5. On the target device, go to /etc/data/, find the tensor
bins. To pull the bins from the target device to the Linux host computer:
- For Qualcomm Linux, run the following
commands:
scp root@:/etc/data/tensor_0.bin .Copy to clipboard
scp root@:/etc/data/tensor_1.bin .Copy to clipboard
scp root@:/etc/data/tensor_2.bin .Copy to clipboard
scp root@:/etc/data/tensor_3.bin .Copy to clipboard
scp root@:/etc/data/tensor_4.bin .Copy to clipboard
- For Ubuntu Server use the following reference
command:
scp ubuntu@:/etc/data/tensor_.binCopy to clipboard
6. To merge the tensor bins with all the facial properties into a cohesive image,
download and run the facedb.pyscript in the same directory
as the tensor bins on the Linux host computer.
1. Download the facedb.py
script:
curl -L -O https://raw.githubusercontent.com/quic/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/facedb.pyCopy to clipboard
2. Run the script. Note that <Name of the person> is
case and style sensitive. Ensure that you use the same name
consistently.
python3 ./facedb.py "" 512 32 tensor_0.bin tensor_1.bin tensor_2.bin tensor_3.bin tensor_4.binCopy to clipboard
A face.bin binary is created.
7. Push the face.bin binary to
/etc/data directory and rename it to
face0.bin.
scp face.bin root@:/etc/data/face0.binCopy to clipboard
For Ubuntu Server, copy the `face.bin` binary to the user home
directory and then use the `sudo` command to copy it to the
`/etc/models` directory.
8. To generate the face\_recognition.labels file and register
the new person into the database, use the following reference label file for
two-person registered
face:
(structure)",id=(guint)0x0,database=(string)/etc/data/face0.bin,color=(guint)0x00FF00FF;"Copy to clipboard
(structure)",id=(guint)0x01,database=(string)/etc/data/face1.bin,color=(guint)0x00FF00FF;"Copy to clipboard
Note: Update the ID field according to the number in the
list. If more faces are registered, add the structure in a new line within
face\_recognition.labels.
9. To save the labels, push the updated
face\_recognition.labels file to the
/etc/labels directory on the target
device.
scp face_recognition.labels root@:/etc/labelsCopy to clipboard
For
Ubuntu Server, copy the `face_recognition.labels` file to the
user `home` folder and then use the `sudo`
command to copy it to the `/etc/labels` directory.
## Run the application on the target device
Note: The following commands provide the default model and label
paths. If you have a different folder structure, replace the default paths in the
command-line parameters. See [Sample model and label files](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_bxr_x4m_qdc).
The sample application uses the
`/etc/configs/config-face-recognition.json` file to read the
input parameters.
To create your own config JSON file, use [config-face-recognition.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/tree/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-face-recognition?ref_type=heads)as a
reference.
1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-70020-50/topic/download-model-and-label-files.html).
2. Update the config JSON file based on the model, input stream, and other
properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_qjx_hqq_32c).
3. Use the following format of the
`/etc/configs/config-face-recognition.json`
file:
{
"ml-framework": "",
"face-detection-model": "",
"face-landmark-model":"",
“face-recognition-model”:””,
"face-detection-labels":””,
"face-landmark-labels":””,
"face-recognition-labels":””,
"face-detection-constants":””,
"face-landmark-constants":””,
"face-recognition-constants":””
}Copy to clipboard
For example, run the application using LiteRT, custom
models, custom labels
file:
{
"ml-framework":"tflite",
"face-detection-model":"/etc/models/face_det_lite_quantized.tflite",
"face-landmark-model":"/etc/models/facemap_3dmm_quantized.tflite",
"face-recognition-model":"/etc/models/face_attrib_net_quantized.tflite",
"face-detection-labels": "/etc/labels/face_detection.labels",
"face-landmark-labels": "/etc/labels/face_landmark.labels",
"face-recognition-labels": "/etc/labels/face_recognition.labels",
"face-detection-constants": "DET,q-offsets=<178.0, 0.0, 102.0>,q-scales=<0.03400895744562149, 0.21995200216770172, 0.1414264440536499>;",
"face-landmark-constants":"DMM,q-offsets=<211.0>,q-scales=<0.06002333015203476>;",
"face-recognition-constants":"qfr,q-offsets=<124.0, 153.0, 125.0, 133.0, 126.0, 0.0>, q-scales=<0.1948956549167633, 0.00791067536920309, 0.06736132502555847, 0.029019491747021675, 0.08928389847278595, 0.00390625>;"
}Copy to clipboard
4. Run the gst-ai-face-recognition
application:
gst-ai-face-recognition --config-file=/etc/configs/config-face-recognition.jsonCopy to clipboard
5. To display the available help options, run the following commands in the SSH
shell:
gst-ai-face-recognition -hCopy to clipboard
6. To stop the use case, use CTRL +
C.
## Expected output
## Pipeline flow
The following table lists the plugins used in the face recognition
pipeline:| Plugin | Description |
| --- | --- |
| Camera source:[qtiqmmfsrc](https://docs.qualcomm.com/doc/80-70020-50/topic/qtiqmmfsrc.html) |
Captures the live stream from camera.
Uses tee to split the stream for inferencing.
|
| File source: filesrc |
Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.
Uses tee to split the stream for inferencing.
|
| RTSP source: rtspsrc |
Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.
Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70020-50/topic/v4l2h264dec.html) | Decodes the video. |
| [qtimlvconverter](https://docs.qualcomm.com/doc/80-70020-50/topic/qtimlvconverter.html) |
Receives the video stream on its sink pad.
Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.
Color conversion
Scaling (up or down)
Normalization
Converts the preprocessed video stream to a tensor stream on its source pad.
The tensor stream is used for inferencing in the later stages of the pipeline. |
| [qtimlqnn](https://docs.qualcomm.com/doc/80-70020-50/topic/qtimlqnn.html) | Acts as the inferencing plugin for Qualcomm Neural Network model.
After the inference runtime receives the tensor stream on its sink pad, it runs the inference.
Produces a tensor stream with the inference results on its source pad.
Receives the output of the face detection models from qtimlvdetection and multiplexes it.
Receives the output of facial pose from qtimlvpose and multiplexes it.
|
| tee | Splits the stream for inferencing. |
| [qtimlvpose](https://docs.qualcomm.com/doc/80-70020-50/topic/qtimlvpose.html) | Performs the facial pose recognition. |
| [qtimlvclassification](https://docs.qualcomm.com/doc/80-70020-50/topic/qtimlvclassification.html) | Receives the stream from qtimetamux and classifies the face. |
| [qtivoverlay](https://docs.qualcomm.com/doc/80-70020-50/topic/qtioverlay.html) |
Waylandsink submits the video stream received on its sink pad to Weston.
Weston renders the video stream on a local display.
|
## Config JSON field description
Table : Field description–config-face-recognition.json file
| Field | Values/description |
| :--- | :--- |
| **ml-framework** | Use one of the following models:
tflite–LiteRT
qnn–Qualcomm AI Engine direct
|
| **Models and labels** | See [Sample model and label files](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-face-recognition.html#gst-ai-face-recognition__section_bxr_x4m_qdc).
face-detection-model: The path to the face detection model.
face-landmark-model: The path to the face landmark model.
face-recognition-model: The path to the face recognition model.
face-detection-labels: The path to the face detection labels.
face-landmark-labels: The path to the face landmark labels.
face-recognition-labels: The path to the face recognition labels.
|
| **Constants** |
face-detection-constants: The face detection model constants.
face-landmark-constants: The face landmark model constants.
face-recognition-constants The face recognition model constants.
|
## Known issues
The 3D facial landmark isn't observed with LiteRT models.
## Related information
- [Object detection](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-object-detection.html)
- [Pose detection](https://docs.qualcomm.com/doc/80-70020-50/topic/gst-ai-pose-detection.html)
**Parent Topic:** [Run AI/ML sample applications](https://docs.qualcomm.com/doc/80-70020-50/topic/ai-ml-sample-applications.html)
Last Published: Jan 30, 2026
[Previous Topic
Face detection](https://docs.qualcomm.com/bundle/publicresource/80-70020-50/topics/gst-ai-face-detection.md) [Next Topic
Audio classification](https://docs.qualcomm.com/bundle/publicresource/80-70020-50/topics/audio-classification.md)