# Machine learning APIs
Configure the tensor-specific requirements and machine learning properties.
## GstMLType
| Enumeration | Description |
| --- | --- |
| GST\_ML\_TYPE\_UNKNOWN | Invalid data |
| GST\_ML\_TYPE\_INT8 | Data is represented as 1 byte of signed integer value |
| GST\_ML\_TYPE\_UINT8 | Data is represented as 1 byte of unsigned integer value |
| GST\_ML\_TYPE\_INT32 | Data is represented as 4 bytes of signed integer value |
| GST\_ML\_TYPE\_UINT32 | Data is represented as 4 bytes of unsigned integer value |
| GST\_ML\_TYPE\_FLOAT16 | Data is represented as 2 bytes of floating-point value |
| GST\_ML\_TYPE\_FLOAT32 | Data is represented as 4 bytes of floating-point value |
The following table lists the possible values describing the tensor format:
The following list provides information on each of the GstMLType APIs:
- **gst\_ml\_type\_get\_size**
Returns the size of the GstMLType in bytes.
**Prototype**
guint
gst_ml_type_get_size (GstMLType type)
Copy to clipboard
**Parameters**
| [in] | `type` | A GstMLType |
| --- | --- | --- |
**Returns**
The size in bytes.
- **gst\_ml\_type\_from\_string**
Returns GstMLType based on its string version.
**Prototype**
GstMLType
gst_ml_type_from_string (const gchar * type)
Copy to clipboard
**Parameters**
| [in] | `type` | A char string version of a GstMLType |
| --- | --- | --- |
**Returns**
A GstMLType
- **gst\_ml\_type\_to\_string**
Returns a char string version of a GstMLType.
const gchar *
gst_ml_type_to_string (GstMLType type)
Copy to clipboard
**Parameters**
| [in] | `type` | A string version of a GstMLType |
| --- | --- | --- |
**Returns**
A new char string version of a GstMLType
## GstMLInfo
gst\_ml\_info\_from\_caps() is used to add the information on the ML properties from GstCaps.
Information structure describing ML properties
| Field | Description |
| --- | --- |
| `type (GstMLType)` | Type of the tensors |
| `n_tensors (guint)` | Number of tensors |
| `n_dimensions (guint)` | Number of dimensions for each tensor |
| `tensors (guint)` | Array with tensor dimensions |
The following list provides information on each of the GstMLInfo APIs:
- **gst\_ml\_info\_init**
Initializes the GstMLInfo fields.
**Prototype**
void
gst_ml_info_init (GstMLInfo * info)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| --- | --- | --- |
**Returns**
None
- **gst\_ml\_info\_new**
Copies a GstMLInfo structure.
**Prototype**
GstMLInfo *
gst_ml_info_new (void)
Copy to clipboard
**Returns**
A new GstMLInfo. Free with gst\_ml\_info\_free().
- gst\_ml\_info\_copy: Copies a GstMLInfo structure.
**Prototype**
GstMLInfo *
gst_ml_info_copy (const GstMLInfo * info)
Copy to clipboard
| [in] | `info` | A GstMLInfo |
| --- | --- | --- |
**Returns**
A new GstMLInfo. Free with gst\_ml\_info\_free().
- **gst\_ml\_info\_free**
Frees a GstMLInfo structure previously allocated with gst\_ml\_info\_new() or gst\_ml\_info\_copy().
**Prototype**
void
gst_ml_info_free (const GstMLInfo * info)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| --- | --- | --- |
**Returns**
None
- **gst\_ml\_info\_from\_caps**
Parses the capabilities (caps) and updates the `caps` information.
**Prototype**
gboolean
gst_ml_info_from_caps (GstMLInfo * info, const GstCaps * caps)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| --- | --- | --- |
| [in] | `caps` | A GstCaps |
**Returns**
TRUE if the operation was successful
- **gst\_ml\_info\_to\_caps**
Converts the values of info into a GstCaps.
**Prototype**
GstCaps *
gst_ml_info_to_caps (const GstMLInfo * info)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| --- | --- | --- |
**Returns**
A new [GstCaps]([https://gstreamer.freedesktop.org/documentation/gstreamer/gstcaps.html](https://gstreamer.freedesktop.org/documentation/gstreamer/gstcaps.html)) containing the information.
- **gst\_ml\_info\_is\_equal**
Compares two GstMLInfo and returns whether they’re equal.
**Prototype**
gboolean
gst_ml_info_is_equal (const GstMLInfo * l_info, const GstMLInfo * r_info)
Copy to clipboard
**Parameters**
| [in] | `l_info` | A GstMLInfo |
| --- | --- | --- |
| [in] | `r_info` | A GstMLInfo |
**Returns**
TRUE if the operation was successful.
- **gst\_ml\_info\_tensor\_size**
Calculates the size of the tensor specified by its index from GstMLInfo.
**Prototype**
gsize
gst_ml_info_tensor_size (const GstMLInfo * info, guint index)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| --- | --- | --- |
| [in] | `index` | The tensor index |
**Returns**
The size in bytes.
- **gst\_ml\_info\_size**
Calculates the total size of all the tensors inside GstMLInfo.
**Prototype**
gsize
gst_ml_info_size (const GstMLInfo * info)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| --- | --- | --- |
**Returns**
The size in bytes.
## GstMLTensorMeta
These APIs are based on [GstMeta](https://gstreamer.freedesktop.org/documentation/gstreamer/gstmeta.html?gi-language=c), which is a member of the [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) metadata structure.
GstMLTensorMeta information structure
| Field | Description |
| --- | --- |
| meta ([GstMeta](https://gstreamer.freedesktop.org/documentation/gstreamer/gstmeta.html?gi-language=c)) | GstMeta is the parent |
| id (guint) | ID corresponding to the memory index in [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) |
| type (GstMLType) | Tensor type |
| n\_dimensions (guint) | Number of tensor dimensions |
| dimensions (guint) | Array of tensor dimensions |
The following list provides information on each of the GstMLTensorMeta APIs:
- **gst\_buffer\_add\_ml\_tensor\_meta**
Attaches GstMLTensorMeta metadata to a buffer with the given parameters.
**Prototype**
GstMLTensorMeta *
gst_buffer_add_ml_tensor_meta (GstBuffer * buffer, const GstMLType type, const guint n_dimensions, const guint dimensions[GST_ML_TENSOR_MAX_DIMS])
Copy to clipboard
**Parameters**
| [in] | `buffer` | A [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) |
| --- | --- | --- |
| [in] | `type` | The tensor type. |
| [in] | `n_dimensions` | The number of tensor dimensions. |
| [in] | `dimensions` | The array that has the tensor dimensions. |
**Returns**
The GstMLTensorMeta on buffer. Don't free after the code is run.
- **gst\_buffer\_get\_ml\_tensor\_meta**
Gets the first GstMLTensorMeta on buffer.
**Prototype**
GstMLTensorMeta *
gst_buffer_get_ml_tensor_meta (GstBuffer * buffer)
Copy to clipboard
**Parameters**
| [in] | `buffer` | A [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) |
| --- | --- | --- |
**Returns**
The GstMLTensorMeta on buffer. Don't free after the code is run.
- **gst\_buffer\_get\_ml\_tensor\_meta\_id**
Finds the GstMLTensorMeta on buffer with the given ID.
**Prototype**
GstMLTensorMeta *
gst_buffer_get_ml_tensor_meta_id (GstBuffer * buffer, guint id)
Copy to clipboard
**Parameters**
| [in] | `buffer` | A [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) |
| --- | --- | --- |
| [in] | `id` | A metadata ID |
**Returns**
The GstMLTensorMeta on buffer. Don't free after the code is run.
- **gst\_ml\_meta\_tensor\_size**
Gets the total size of the tensor in bytes, which is calculated based on the dimensions and tensor type.
**Prototype**
gsize
gst_ml_meta_tensor_size (const GstMLTensorMeta * meta)
Copy to clipboard
**Parameters**
| [in] | `meta` | A GstMLTensorMeta |
| --- | --- | --- |
**Returns**
The size in bytes.
## GstMLFrame
GstMLFrame is a structure obtained from gst\_ml\_frame\_map().
Structure of GstMLFrame
| Field | Description |
| --- | --- |
| info (GstMLInfo) | The GstMLInfo |
| buffer ([GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c)) | Mapped buffer containing the tensor memory blocks |
| map ([GstMapInfo](https://gstreamer.freedesktop.org/documentation/gstreamer/gstmemory.html?gi-language=c#GstMapInfo)) | Mappings of the tensor memory blocks |
The following list provides information on each of the GstMLFrame APIs:
- **gst\_ml\_frame\_map**
Updates the frame values using [GstMLInfo](https://docs.qualcomm.com/doc/80-80021-50/topic/machine-learning-apis.html#section-qqy-3ng-tcc) and GstBuffer.
1. The address is passed to the GstMLFrame structure that’s allocated on the stack.
2. The gst\_ml\_frame\_map() function fills the GstMLFrame structure with ML tensor-specific information to access the ML buffer data. All the buffer video tensors are mapped and the pointers are set in the frame data.
**Prototype**
gboolean
gst_ml_frame_map (GstMLFrame * frame, const GstMLInfo * info, GstBuffer * buffer, GstMapFlags flags)
Copy to clipboard
**Parameters**
| [in] | `frame` | Pointer to GstMLFrame |
| --- | --- | --- |
| [in] | `info` | A GstMLInfo |
| [in] | `buffer` | A GstBuffer |
| [in] | `flags` | A GstMapFlags |
**Returns**
>
>
> TRUE if the map operation was successful.
- **gst\_ml\_frame\_unmap**:
Un-maps the memory that was earlier mapped with gst\_ml\_frame\_map().
**Prototype**
void
gst_ml_frame_unmap (GstMLFrame * frame)
Copy to clipboard
**Parameters**
| [in] | `frame` | Pointer to GstMLFrame |
| --- | --- | --- |
**Returns**
>
>
> None
## GstMLBufferPool
Configure the following using GstMLBufferPool API:
- Tensor-specific requirements such as several [GstMemory](https://gstreamer.freedesktop.org/documentation/gstreamer/gstmemory.html?gi-language=c) blocks in a single [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c)
Type of memory used for allocation.
- Addition of GstMLTensorMeta to the buffers.
**Figure : GstBufferPool subclass for ML**
- **Prototype**
Creates a buffer pool that can assign the ML frames.
GstBufferPool *
gst_ml_buffer_pool_new (const gchar * type)
Copy to clipboard
- **Parameters**
| [in] | `type` | The memory used by the pool to assign buffers. |
| --- | --- | --- |
- **Returns**
- A new [GstBufferPool](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbufferpool.html?gi-language=c) to assign ML frames.
- Free with [gst_object_unref](https://gstreamer.freedesktop.org/documentation/gstreamer/gstobject.html#gst_object_unref).
- **Arguments**
| **Argument** | **Description** | **Usage** |
| --- | --- | --- |
| GST\_ML\_BUFFER\_POOL\_TYPE\_ION | Assigns buffers by passing it as arguments to gst\_ml\_buffer\_pool\_new() | #define GST_ML_BUFFER_POOL_TYPE_ION "GstMLBufferPoolTypeIonMemory" Copy to clipboard |
| GST\_ML\_BUFFER\_POOL\_TYPE\_SYSTEM | Assigns buffers by passing it as arguments to gst\_ml\_buffer\_pool\_new() | #define GST_ML_BUFFER_POOL_TYPE_SYSTEM "GstMLBufferPoolTypeSystemMemory" Copy to clipboard |
| GST\_ML\_BUFFER\_POOL\_OPTION\_TENSOR\_META | Requests ML tensor metadata on buffers from the pool. | #define GST_ML_BUFFER_POOL_OPTION_TENSOR_META "GstMLBufferPoolOptionTensorMeta" Copy to clipboard |
| GST\_ML\_BUFFER\_POOL\_OPTION\_CONTINUOUS | Requests all tensors to be into a continuous physical memory. | #define GST_ML_BUFFER_POOL_OPTION_CONTINUOUS "GstMLBufferPoolOptionContinuous" Copy to clipboard |
| | | |
The following table lists arguments that can be passed to gst\_ml\_buffer\_pool\_new() to configure the buffer memories:
Last Published: Mar 26, 2026
[Previous Topic
Video APIs](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/video-apis.md) [Next Topic
GSt debug](https://docs.qualcomm.com/bundle/publicresource/80-80021-50/topics/imsdk_debug_gst.md)
Source: [https://docs.qualcomm.com/doc/80-80021-50/topic/machine-learning-apis.html](https://docs.qualcomm.com/doc/80-80021-50/topic/machine-learning-apis.html)