# Audio classification The **gst-ai-audio-classification** application shows audio classification using input from either a file source or a microphone. It displays both the classification results and a video preview. Note This application isn't supported in the Config #1 for the QLI 2.0 RC2 release because only the CPU runtime is supported. The following figure shows the pipeline, which gets the input from a file or a microphone, preprocesses it, and runs inferences on AI hardware. The results are displayed on the screen. For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-80021-50/topic/audio-classification.html#section-fdn-fmz-n2c). **Figure : gst-ai-audio-classification pipeline** ## Input and output capabilities The following table summarizes the input and output capabilities supported by the sample application: | Config | Input | Input | Input | Input | Output | Output | Output | | --- | --- | --- | --- | --- | --- | --- | --- | | Config | File src | RTSP | USB camera | MIPI camera | File | Display | RTSP | | Config #2 | Yes | No | No | No | No | Yes | No | | | | | | | | | | ## Sample model and label files Table : Sample model and label files for gst-ai-audio-classification | Runtime | Model files | Label files | | --- | --- | --- | | LiteRT | *yamnet.tflite* | *yamnet.json* | ## Run the application on the target device The sample application uses the `/etc/configs/config-audio-classification.json` file to read the input parameters. To create your own config JSON file, use [config-audio-classification.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-audio-classification/config-audio-classification.json?ref_type=heads) as a reference. 1. Ensure that you complete the [Prerequisites](https://docs.qualcomm.com/doc/80-80021-50/topic/download-model-and-label-files.html). 2. Update the config JSON file based on the model, input stream, and other properties. For more information, see [Config JSON field description](https://docs.qualcomm.com/doc/80-80021-50/topic/gst-ai-classification.html#section-lcw-2zj-32c). 3. Use the following format of the `config-audio-classification.json` file. { "file-path": "", "model": "", "labels": "", "threshold": , "runtime": "", "codec": "" } Copy to clipboard For example, run the application using input from a file, MP3 encoding, and GPU runtime: { "file-path": "/etc/media/video-mp3.mp4", "model": "/etc/models/yamnet.tflite", "labels": "/etc/labels/yamnet.json", "runtime": "cpu", "threshold": 20, "codec": "mp3" } Copy to clipboard 4. Run the gst-ai-audio-classification application: gst-ai-audio-classification --config-file=/etc/configs/config-audio-classification.json Copy to clipboard 5. To display the available help options, run the following command in the SSH shell: gst-ai-audio-classification -h Copy to clipboard 6. To stop the use case, use **CTRL + C**. ## Expected output The output video and classified audio are played on the screen. ## Pipeline flow The following table lists the plugins used in the audio classification pipeline: | Plugin | Description | | --- | --- | | File source: filesrc |

Captures the video stream using filesrc, followed by qtdemux, which demultiplexes the stream.

Uses tee to split the stream for inferencing.

| | h264parse | Parses the H.264 video. | | [v4l2h264dec](https://docs.qualcomm.com/doc/80-80021-50/topic/v4l2h264dec.html) | Decodes the video. | | mpegaudioparse or flacparse | Parses the audio (MP3 or FLAC). | | mpg123audiodec or flacdec | Decodes the audio (MP3 or FLAC). | | audioconvert | Converts the audio buffers between various possible formats. | | audioresample | Resamples the audio buffers to different sample rates. | | [pulsesrc](https://docs.qualcomm.com/doc/80-80021-50/topic/pulsesrc.html) | Reads the audio from the microphone. | | audiobuffersplit | Splits the incoming audio buffers into equal sized chunks. | | [qtimlaconverter](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlaconverter.html) | Performs preprocessing on the audio stream and converts the stream to a tensor stream.

The audio classification model uses this tensor stream for inferencing. | | [qtimltflite](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimltflite.html) | Performs inferencing using the YAMNet model. | | [qtimlpostprocess](https://docs.qualcomm.com/doc/80-80021-50/topic/qtimlpostprocess.html) | Uses yamnet module to handle the audio classification inference results:

Applies a threshold to the chosen number of results.

Creates text overlay for classes.

| | [qtivcomposer](https://docs.qualcomm.com/doc/80-80021-50/topic/qtivcomposer.html) | Combines the text overlay for classification results and video preview. | | [Waylandsink](https://docs.qualcomm.com/doc/80-80021-50/topic/waylandsink.html) |

Waylandsink submits the video stream received on its sink pad to Weston.

Weston renders the video stream on a local display.

| ## Config JSON field description The different parameters available to configure the JSON file and run the use case are as follows: | Field | Values/description | | --- | --- | | **runtime** | Use one of the following runtimes:

cpu

gpu

| | **Input source** | Use one of the following input sources:

file-path: The directory path to the video file.

Microphone

MP3 (default)

FLAC

| ## Known issue The FLAC format isn't supported in the Config #2. ## Related information [Audio classification decode and display with LiteRT](https://docs.qualcomm.com/doc/80-80021-50/topic/audio-classification-with-litert.html) Last Published: Mar 26, 2026 [Previous Topic Face recognition](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/gst-ai-face-recognition.md) [Next Topic Metadata parsing](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/gst-ai-metadata-parser.md) Source: [https://docs.qualcomm.com/doc/80-80021-50/topic/audio-classification.html](https://docs.qualcomm.com/doc/80-80021-50/topic/audio-classification.html)