# Machine learning APIs
Source: [https://docs.qualcomm.com/doc/80-70015-50/topic/machine-learning-apis.html](https://docs.qualcomm.com/doc/80-70015-50/topic/machine-learning-apis.html)
Configure the tensor-specific requirements and machine learning
properties.
## GstMLType
The following are the possible values describing the tensor format:
| Enumeration | Description |
| --- | --- |
| GST\_ML\_TYPE\_UNKNOWN | Invalid data |
| GST\_ML\_TYPE\_INT8 | Data is represented as 1 byte of signed integer value |
| GST\_ML\_TYPE\_UINT8 | Data is represented as 1 byte of unsigned integer value |
| GST\_ML\_TYPE\_INT32 | Data is represented as 4 bytes of signed integer value |
| GST\_ML\_TYPE\_UINT32 | Data is represented as 4 bytes of unsigned integer value |
| GST\_ML\_TYPE\_FLOAT16 | Data is represented as 2 bytes of floating-point value |
| GST\_ML\_TYPE\_FLOAT32 | Data is represented as 4 bytes of floating-point value |
The following list provides information on each of the GstMLType APIs:
- **gst\_ml\_type\_get\_size**
Returns the size of the GstMLType in
bytes.
**Prototype**
guint
gst_ml_type_get_size (GstMLType type)
Copy to clipboard
**Parameters**
| [in] | `type` | A GstMLType |
| :--- | :--- | :--- |
**Returns**
The size in bytes.
- **gst\_ml\_type\_from\_string**
Returns GstMLType based on its string
version.
**Prototype**
GstMLType
gst_ml_type_from_string (const gchar * type)
Copy to clipboard
**Parameters**
| [in] | `type` | A char string version of a GstMLType |
| :--- | :--- | :--- |
**Returns**
A GstMLType
- **gst\_ml\_type\_to\_string**
Returns a char string version of a
GstMLType.
const gchar *
gst_ml_type_to_string (GstMLType type)
Copy to clipboard
**Parameters**
| [in] | `type` | A string version of a GstMLType |
| :--- | :--- | :--- |
**Returns**
A new char string version of a
GstMLType
## GstMLInfo
gst\_ml\_info\_from\_caps() is used to add the information on the ML
properties from GstCaps.
Table : Information structure describing ML properties
| Field | Description |
| --- | --- |
| `type (GstMLType)` | Type of the tensors |
| `n_tensors (guint)` | Number of tensors |
| `n_dimensions (guint)` | Number of dimensions for each tensor |
| `tensors (guint)` | Array with tensor dimensions |
The following list provides information on each of the GstMLInfo APIs:
- **gst\_ml\_info\_init**
Initializes the GstMLInfo
fields.
**Prototype**
void
gst_ml_info_init (GstMLInfo * info)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| :--- | :--- | :--- |
**Returns**
None
- **gst\_ml\_info\_new**
Copies a GstMLInfo
structure.
**Prototype**
GstMLInfo *
gst_ml_info_new (void)Copy to clipboard
**Returns**
A new
GstMLInfo. Free with gst\_ml\_info\_free().
- gst\_ml\_info\_copy: Copies a GstMLInfo
structure.
**Prototype**
GstMLInfo *
gst_ml_info_copy (const GstMLInfo * info)
Copy to clipboard
| [in] | `info` | A GstMLInfo |
| :--- | :--- | :--- |
**Returns**
A new GstMLInfo. Free with
gst\_ml\_info\_free().
- **gst\_ml\_info\_free**
Frees a GstMLInfo structure previously allocated with
gst\_ml\_info\_new() or
gst\_ml\_info\_copy().
**Prototype**
void
gst_ml_info_free (const GstMLInfo * info)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| :--- | :--- | :--- |
**Returns**
None
- **gst\_ml\_info\_from\_caps**
Parses the capabilities (caps) and updates the
`caps`
information.
**Prototype**
gboolean
gst_ml_info_from_caps (GstMLInfo * info, const GstCaps * caps)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| :--- | :--- | :--- |
| [in] | `caps` | A GstCaps |
**Returns**
TRUE if the operation was successful
- **gst\_ml\_info\_to\_caps**
Converts the values of info into a
GstCaps.
**Prototype**
GstCaps *
gst_ml_info_to_caps (const GstMLInfo * info)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| :--- | :--- | :--- |
**Returns**
A new [GstCaps](https://gstreamer.freedesktop.org/documentation/gstreamer/gstcaps.html) containing the
information.
- **gst\_ml\_info\_is\_equal**
Compares two GstMLInfo and returns whether they are
equal.
**Prototype**
gboolean
gst_ml_info_is_equal (const GstMLInfo * l_info, const GstMLInfo * r_info)
Copy to clipboard
**Parameters**
| [in] | `l_info` | A GstMLInfo |
| :--- | :--- | :--- |
| [in] | `r_info` | A GstMLInfo |
**Returns**
TRUE if the operation was successful.
- **gst\_ml\_info\_tensor\_size**
Calculates the size of the tensor specified by
its index from
GstMLInfo.
**Prototype**
gsize
gst_ml_info_tensor_size (const GstMLInfo * info, guint index)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| :--- | :--- | :--- |
| [in] | `index` | The tensor index |
**Returns**
The size in bytes.
- **gst\_ml\_info\_size**
Calculates the total size of all the tensors inside
GstMLInfo.
**Prototype**
gsize
gst_ml_info_size (const GstMLInfo * info)
Copy to clipboard
**Parameters**
| [in] | `info` | A GstMLInfo |
| --- | --- | --- |
**Returns**
The size in bytes.
## GstMLTensorMeta
These APIs are based on [GstMeta](https://gstreamer.freedesktop.org/documentation/gstreamer/gstmeta.html?gi-language=c), which is a member of the [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) metadata structure.
Table : GstMLTensorMeta information structure
| Field | Description |
| --- | --- |
| meta ([GstMeta](https://gstreamer.freedesktop.org/documentation/gstreamer/gstmeta.html?gi-language=c)) | GstMeta is the parent |
| id (guint) | ID corresponding to the memory index in [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) |
| type (GstMLType) | Tensor type |
| n\_dimensions (guint) | Number of tensor dimensions |
| dimensions (guint) | Array of tensor dimensions |
The following list provides information on each of the GstMLTensorMeta APIs:
- **gst\_buffer\_add\_ml\_tensor\_meta**
Attaches GstMLTensorMeta metadata to a
buffer with the given
parameters.
**Prototype**
GstMLTensorMeta *
gst_buffer_add_ml_tensor_meta (GstBuffer * buffer, const GstMLType type, const guint n_dimensions, const guint dimensions[GST_ML_TENSOR_MAX_DIMS])Copy to clipboard
**Parameters**
| [in] | `buffer` | A [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) |
| :--- | :--- | :--- |
| [in] | `type` | The tensor type. |
| [in] | `n_dimensions` | The number of tensor dimensions. |
| [in] | `dimensions` | The array that contains the tensor dimensions. |
**Returns**
The GstMLTensorMeta on buffer. Do not free after
the code is run.
- **gst\_buffer\_get\_ml\_tensor\_meta**
Gets the first GstMLTensorMeta on
buffer.
**Prototype**
GstMLTensorMeta *
gst_buffer_get_ml_tensor_meta (GstBuffer * buffer)Copy to clipboard
**Parameters**
| [in] | `buffer` | A [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) |
| :--- | :--- | :--- |
**Returns**
The GstMLTensorMeta on buffer. Do not free after
the code is run.
- **gst\_buffer\_get\_ml\_tensor\_meta\_id**
Finds the GstMLTensorMeta on buffer
with the given
ID.
**Prototype**
GstMLTensorMeta *
gst_buffer_get_ml_tensor_meta_id (GstBuffer * buffer, guint id)
Copy to clipboard
**Parameters**
| [in] | `buffer` | A [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c) |
| :--- | :--- | :--- |
| [in] | `id` | A metadata ID |
**Returns**
The GstMLTensorMeta on buffer. Do not free after
the code is run.
- **gst\_ml\_meta\_tensor\_size**
Gets the total size of the tensor in bytes,
which is calculated based on the dimensions and tensor
type.
**Prototype**
gsize
gst_ml_meta_tensor_size (const GstMLTensorMeta * meta)Copy to clipboard
**Parameters**
| [in] | `meta` | A GstMLTensorMeta |
| :--- | :--- | :--- |
**Returns**
The size in bytes.
## GstMLFrame
GstMLFrame is a structure obtained from gst\_ml\_frame\_map().
Table : Structure of GstMLFrame
| Field | Description |
| --- | --- |
| info (GstMLInfo) | The GstMLInfo |
| buffer ([GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c)) | Mapped buffer containing the tensor memory blocks |
| map ([GstMapInfo](https://gstreamer.freedesktop.org/documentation/gstreamer/gstmemory.html?gi-language=c#GstMapInfo)) | Mappings of the tensor memory blocks |
The following list provides information on each of the GstMLFrame APIs:
- **gst\_ml\_frame\_map**
Updates the frame values using [GstMLInfo](https://docs.qualcomm.com/doc/80-70015-50/topic/machine-learning-apis.html#machine-learning-apis__section_qqy_3ng_tcc) and
GstBuffer.
1. The address is passed to the GstMLFrame structure that is allocated on
the stack.
2. The gst\_ml\_frame\_map() function fills the GstMLFrame
structure with ML tensor-specific information to access the ML buffer
data. All the buffer video tensors are mapped and the pointers are set
in the frame data.
**Prototype**
gboolean
gst_ml_frame_map (GstMLFrame * frame, const GstMLInfo * info, GstBuffer * buffer, GstMapFlags flags)
Copy to clipboard
**Parameters**
| [in] | `frame` | Pointer to GstMLFrame |
| :--- | :--- | :--- |
| [in] | `info` | A GstMLInfo |
| [in] | `buffer` | A GstBuffer |
| [in] | `flags` | A GstMapFlags |
**Returns**
TRUE if the map operation was
successful.
- **gst\_ml\_frame\_unmap**:
Un-maps the memory that was previously mapped with
gst\_ml\_frame\_map().
**Prototype**
void
gst_ml_frame_unmap (GstMLFrame * frame)Copy to clipboard
**Parameters**
| [in] | `frame` | Pointer to GstMLFrame |
| :--- | :--- | :--- |
**Returns**
None
## GstMLBufferPool
Configure the following using GstMLBufferPool API:
- Tensor-specific requirements such as multiple [GstMemory](https://gstreamer.freedesktop.org/documentation/gstreamer/gstmemory.html?gi-language=c) blocks in a single [GstBuffer](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c)
Type of memory used
for allocation.
- Addition of GstMLTensorMeta to the buffers.
Figure : GstBufferPool subclass for ML
- **Prototype**
Creates a buffer pool that can allocate the ML
frames.
GstBufferPool *
gst_ml_buffer_pool_new (const gchar * type)
Copy to clipboard
- **Parameters**
| [in] | `type` | The memory used by the pool to allocate buffers. |
| :--- | :--- | :--- |
- **Returns**
- A new [GstBufferPool](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbufferpool.html?gi-language=c) to allocate ML
frames.
- Free with [gst_object_unref](https://gstreamer.freedesktop.org/documentation/gstreamer/gstobject.html#gst_object_unref).
- **Arguments**
The following arguments can be passed to
gst\_ml\_buffer\_pool\_new() to configure the buffer
memories:
| Argument | Description | Usage |
| --- | --- | --- |
| GST\_ML\_BUFFER\_POOL\_TYPE\_ION | Allocates buffers by passing it as
arguments to
gst\_ml\_buffer\_pool\_new() | #define GST_ML_BUFFER_POOL_TYPE_ION "GstMLBufferPoolTypeIonMemory"Copy to clipboard |
| GST\_ML\_BUFFER\_POOL\_TYPE\_SYSTEM | Allocates buffers by passing it as
arguments to
gst\_ml\_buffer\_pool\_new() | #define GST_ML_BUFFER_POOL_TYPE_SYSTEM "GstMLBufferPoolTypeSystemMemory"Copy to clipboard |
| GST\_ML\_BUFFER\_POOL\_OPTION\_TENSOR\_META | Requests ML tensor metadata on buffers from the
pool. | #define GST_ML_BUFFER_POOL_OPTION_TENSOR_META "GstMLBufferPoolOptionTensorMeta"Copy to clipboard |
| GST\_ML\_BUFFER\_POOL\_OPTION\_CONTINUOUS | Requests all tensors to be into a continuous physical
memory. | #define GST_ML_BUFFER_POOL_OPTION_CONTINUOUS "GstMLBufferPoolOptionContinuous"Copy to clipboard |
| | | |
**Parent Topic:** [GSt APIs](https://docs.qualcomm.com/doc/80-70015-50/topic/gst-apis.html)
Last Published: Oct 27, 2025
[Previous Topic
Video APIs](https://docs.qualcomm.com/bundle/publicresource/80-70015-50/topics/video-apis.md) [Next Topic
GSt debugging tools](https://docs.qualcomm.com/bundle/publicresource/80-70015-50/topics/imsdk_debug_gst.md)