# Audio classification
Source: [https://docs.qualcomm.com/doc/80-70018-50/topic/audio-classification.html](https://docs.qualcomm.com/doc/80-70018-50/topic/audio-classification.html)
The **gst-ai-audio-classification** application shows the audio classification
using the input from a file source and microphone. The classification results and video
preview are displayed.
The figure shows the pipeline, which gets the input from a file or a microphone, does the
preprocessing and inferencing on the AI hardware. The results are displayed on the
screen.
For information about the plugins used in the pipeline flow, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70018-50/topic/audio-classification.html#audio-classification__section_fdn_fmz_n2c).
Figure : gst-ai-audio-classification pipeline
## Sample model and label files
Table : Sample model and label files for gst-ai-audio-classification
| Runtime | Model files | Label files |
| :--- | :--- | :--- |
| LiteRT | yamnet.tflite | yamnet.labels |
## Prerequisites
- If not already done so, [Download and install eSDK](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-51/install-sdk.html#download-and-install-esdk-).
- [Download model and label files](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html).
- To access your host computer, enable SSH. For instructions, see [Sign in using SSH](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-254/how_to.html#use-ssh).
Note: If SSH is already enabled, you can skip this
step.
- Push the model files from the Linux
host.
scp root@:/etc/modelsCopy to clipboard
- Note that the [downloaded
script](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html) downloads the sample video.mp4 video to
the /etc/media directory. If you are using a custom video,
then ensure that you push the video to /etc/media and
update the file path in the application config. JSON file.
- Ensure that the video in use has an audio component along with the video.
- Connect the display to the device using the HDMI port. For instructions, see
[Set up HDMI display](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/samples.html).
- Enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
## Run the application
The sample application uses the
/etc/configs/config-audio-classification.json file to read
the input parameters.
To create your own config JSON file, use [config-audio-classification.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-audio-classification/config-audio-classification.json?ref_type=heads) as a
reference.
1. Use the following format of the
config-audio-classification.json file.
{
"file-path": "",
"model": "",
"labels": "",
"threshold": ,
"runtime": "",
"codec": ""
}Copy to clipboard
Note: Update the config JSON file based
on the model, input stream, and other properties. For more information, see
[Config JSON field description](https://docs.qualcomm.com/doc/80-70018-50/topic/audio-classification.html#audio-classification__section_lcw_2zj_32c).
For example, run the application using input from a file, MP3 encoding, and GPU
runtime:
{
"file-path": "/etc/media/video-mp3.mp4",
"model": "/etc/models/yamnet.tflite",
"labels": "/etc/labels/yamnet.labels",
"runtime": "gpu",
"threshold": 20,
"codec": "mp3"
}Copy to clipboard
2. Run the gst-ai-audio-classification
application:
gst-ai-audio-classification --config-file=/etc/configs/config-audio-classification.jsonCopy to clipboard
To display the available help options, run the following command in the SSH
shell:
gst-ai-audio-classification -hCopy to clipboard
To stop the use case, use CTRL + C.
## Expected output
The output video and classified audio are played on the screen.
## Pipeline flow
The table lists the plugins used in the audio classification pipeline:| Plugin | Description |
| --- | --- |
| File source: filesrc |
- Captures the video stream using filesrc, followed by
qtdemux, which demultiplexes the stream.
- Uses tee to split the stream for inferencing.
|
| h264parse | Parses the H.264 video. |
| [v4l2h264dec](https://docs.qualcomm.com/doc/80-70018-50/topic/v4l2h264dec.html) | Decodes the video. |
| mpegaudioparse or flacparse | Parses the audio (MP3 or FLAC). |
| mpg123audiodec or flacdec | Decodes the audio (MP3 or FLAC). |
| audioconvert | Converts the audio buffers between various possible
formats. |
| audioresample | Resamples the audio buffers to different sample rates. |
| [pulsesrc](https://docs.qualcomm.com/doc/80-70018-50/topic/pulsesrc.html) | Reads the audio from the microphone. |
| audiobuffersplit | Splits the incoming audio buffers into equal sized
chunks. |
| [qtimlaconverter](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlaconverter.html) | Performs preprocessing on the audio stream and converts the
stream to a tensor stream.
The audio classification model uses
this tensor stream for inferencing. |
| [qtimltflite](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimltflite.html) | Performs inferencing using the YAMNet model. |
| [qtimlaclassification](https://docs.qualcomm.com/doc/80-70018-50/topic/qtimlaclassification.html) | Handles the audio classification inference results:
- Applies a threshold to the chosen number of results.
- Creates text overlay for classes.
|
| [qtivcomposer](https://docs.qualcomm.com/doc/80-70018-50/topic/qtivcomposer.html) | Combines the text overlay for classification results and video
preview. |
| [Waylandsink](https://docs.qualcomm.com/doc/80-70018-50/topic/waylandsink.html) |
- Waylandsink submits the video stream received on its sink
pad to Weston.
- Weston renders the video stream on a local display.
|
## Config JSON field description
| Field | Values/description |
| :--- | :--- |
| **runtime** | Use one of the following runtimes: |
| **Input source** | Use one of the following input sources:
file-path: The directory path to the video
file.
- Microphone
|
| **threshold=<integer>** | Use any integer between 1 and 100. |
| **codec** | The audio codec of input video: |
**Parent Topic:** [Run AI/ML sample applications](https://docs.qualcomm.com/doc/80-70018-50/topic/ai-ml-sample-applications.html)
**Related Resources**
- [Audio classification decode and display with LiteRT](https://docs.qualcomm.com/doc/80-70018-50/topic/audio-classification-with-litert.html)
Last Published: Jan 30, 2026
[Previous Topic
Face recognition](https://docs.qualcomm.com/bundle/publicresource/80-70018-50/topics/gst-ai-face-recognition.md) [Next Topic
Metadata parsing](https://docs.qualcomm.com/bundle/publicresource/80-70018-50/topics/gst-ai-metadata-parser.md)