# AI event encoder The **gst-ai-event-encoder** application receives the live video stream input from camera, file, or RTSP source. When a human enters the video frame the application preprocesses the video, runs inferences on the AI hardware, and encodes the video. The encoding stops 5 seconds after the human moves away from the frame and restarts when anyone enters the frame. The following figures show the event detection and recording pipelines for event encoder application. For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-event-encoder.html#section-tsy-xsy-pgc). Qualcomm Open source tee filesrc qtdemux h264parse V4l2h264dec qtimlvconverter qtimltflite qtimlpostprocess qtimlpostprocess appsink tee sink_1 video/x-raw text/x-raw sink_0 tee rtspsrc rtph264 depay h264parse V4l2h264dec sink_0 qtimlvconverter qtimltflite qtimlpostprocess qtimlpostprocess appsink tee sink_1 video/x-raw text/x-raw qtivcomposer Waylandsink qtiqmmfsrc sink_0 qtimlvconverter qtimltflite qtimlpostprocess qtimlpostprocess appsink tee sink_1 video/x-raw text/x-raw File (default) Camera (optional) RTSP (optional) Event detection Event detection Event detection **Figure : gst-ai-event-encoder event detection pipeline** Open source appsrc File v4l2h264enc mp4mux filesink **Figure : gst-ai-event-encoder recording pipeline** ## Input and output capabilities The following table summarizes the input and output capabilities supported by the sample application: | Config | Input | Input | Input | Input | Output | Output | Output | | --- | --- | --- | --- | --- | --- | --- | --- | | Config | File src | RTSP | USB camera | MIPI camera | File | Display | RTSP | | Config #1 | Yes | Yes | No | No | No | Yes | No | | Config #2 | Yes | Yes | No | Yes | No | Yes | No | | | | | | | | | | ## Sample model and label files Table : Sample model and label files for gst-ai-event-encoder | Runtime | Model files | Label files | | --- | --- | --- | | LiteRT | *yolox\_quantized.tflite* | *yolox.json* | ## Run the application on the target device Use the following command to run the application: gst-ai-event-encoder --config-file=/etc/configs/config-event-encoder.json Copy to clipboard The sample application uses the `/etc/configs/config-event-encoder.json` file to read the input parameters. To create your own config JSON file, use [config-event-encoder.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-event-encoder/config-event-encoder.json?ref_type=heads) as a reference. 1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-80021-50/topic/download-model-and-label-files.html). 2. Update the config JSON file based on the model, input stream, and other properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-event-encoder.html#section-apm-bty-pgc). 3. Use the following format of the `config-event-encoder.json` file. { "file-path": "", "model": "", "labels": "", "threshold": , "runtime": "" } Copy to clipboard An example format file for Config #2 and Config #1 is as follows: Tab Config #2 Tab Config #1 For example, run the application using input from a camera, YOLOX model, and human count on console: > > > { > "file-path": "/etc/media/video.mp4", > "model": "/etc/models/yolox_quantized.tflite", > "labels": "/etc/labels/yolox.json", > "threshold": 40, > "runtime": "dsp" > } > Copy to clipboard For example, run the application using input from a camera, YOLOX model, and human count on console: > > > { > "file-path": "/etc/media/video.mp4", > "model": "/etc/models/yolox_quantized.tflite", > "labels": "/etc/labels/yolox.json", > "threshold": 40, > "runtime": "cpu" > } > Copy to clipboard > > > Note > > > Config #1 supports only LiteRT models and the CPU runtime. 4. Run the gst-ai-event-encoder application: gst-ai-event-encoder --config-file=/etc/configs/config-event-encoder.json Copy to clipboard 5. To display the available help options, run the following command in the SSH shell: gst-ai-event-encoder -h Copy to clipboard 6. To stop the use case, use **CTRL + C**. ## Expected output The output is saved as an MP4 file within `etc/media` folder as `output-1.mp4`, `output-2.mp4`, and so on. ## Pipeline flow The following table lists the plugins used in the AI event encoder pipeline: | **Plugin** | **Description** | | --- | --- | | Camera source: [qtiqmmfsrc](https://docs.qualcomm.com/doc/80-80021-50/topic/qtiqmmfsrc.html) |

  • Captures the live stream from camera.


  • Uses tee to split the stream into two for inferencing and composing.


| | File source: filesrc |

  • Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.


  • Uses tee to split the stream into two for inferencing and composing.


| | RTSP source: rtspsrc |

  • Captures the RTSP stream using rtspsrc, followed by rtph264depay for video extraction.


  • Uses tee to split the stream into two for inferencing and composing.


| | h264parse | Parses the H.264 video. | | [v4l2h264dec](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264dec.html) | Decodes the video. | | [qtimlvconverter](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlvconverter.html) |

  1. Receives the video stream on its sink pad.


  2. Performs the following preprocessing on the stream data. This preprocessing is done when the model expects floating-point values as input.


    1. Color conversion


    2. Scaling (up or down)


    3. Normalization




  3. Converts the preprocessed video stream to a tensor stream on its source pad.




The tensor stream is used for inferencing in the later stages of the pipeline. | | Inferencing plugin: [qtimltflite](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimltflite.html) |

  1. After the inference runtime receives the tensor stream on its sink pad, it runs the inference.


  2. Produces a tensor stream with the inference results on its source pad.


| | [qtimlpostprocess](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlpostprocess.html) | The first qtimlpostprocess plugin for detection model handles inference results from any object detection model.

  1. Applies a threshold to the chosen number of results.


  2. Loads the YOLOv8 module.


  3. Produces video frames with only bounding boxes that can be overlaid on objects.


  4. Sends these processed frames to the sink pad of qtivcomposer.


| | [qtimlpostprocess](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlpostprocess.html) | The second qtimlpostprocess plugin for detection model does the following:

  • Produces the output in a text format (bounding box coordinates and labels).


  • This output is connected to appsink plugin where the metadata is read, parsed, and logged.


  • The bounding box information is used to count number of humans in each frame.


| | [qtivcomposer](https://docs.qualcomm.com/doc/80-80021-50/topic/qtivcomposer.html) |

  1. Composes frames with contents from its sink pads.


  2. Pushes the GStreamer buffers containing these composed frames to its source pad.


| | [Waylandsink](https://docs.qualcomm.com/doc/80-80021-50/topic/waylandsink.html) |

  1. Waylandsink submits the video stream received on its sink pad to Weston.


  2. Weston renders the video stream on a local display.


| | | | ## Config JSON field description The different parameters available to configure the JSON file and run the use case are as follows: Table : Field description–gst-ai-event-encoder file | Field | Values/description | | --- | --- | | **runtime** | Use one of the following runtimes:

  • cpu


  • gpu


  • dsp


| | **Input source** | Use one of the following input sources:

  • camera–Primary (0) or secondary (1).


  • file-path–The directory path to the video file.


  • rtsp-ip-port–The address of the RTSP stream: rtsp://<ip>:<port>/<stream>


| ## Known issues Tab Config #2 Tab Config #1 - Lag is observed in encoding with camera source and file source as quantized models aren't supported in the TFlite IM SDK framework. - The application may intermittently hang during the `gst_deinit` phase. - The GPU delegate doesn't function in the QNN and TFLite IM SDK frameworks. This sample application isn’t supported on QCS6490. ## Related information [Object detection](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-object-detection.html) Last Published: Mar 26, 2026 [Previous Topic Metadata parsing](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/gst-ai-metadata-parser.md) [Next Topic AI USB camera](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/ai-usb-camera.md) Source: [https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-event-encoder.html](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-event-encoder.html)