# Region Of Interest Encoding
Table of Contents
- [Region Of Interest Encoding](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#region-of-interest-encoding)
- [Visual Quality Improvement with ROI](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#visual-quality-improvement-with-roi)
- [Enabling ROI Encoding on Snapdragon](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#enabling-roi-encoding-on-snapdragon)
- [ROI extensions](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#roi-extensions)
- [Configuring ROI-type](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#configuring-roi-type)
- [Configuring ROI Rectangles and QP Bias values](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#configuring-roi-rectangles-and-qp-bias-values)
- [Configuring timestamps and queueing the input frame](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#configuring-timestamps-and-queueing-the-input-frame)
- [Query capabilities](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#query-capabilities)
When encoding video, applications must consider both compression ratio and quality for optimal user experience. A higher compression ratio will
result in a smaller file size, which saves storage and bandwidth for the user at the cost of image quality. In most mobile video sharing use cases, it may be a good trade off.
However, the encoded frame could lack key details on the main subjects, such as faces or focused text, resulting in poor user experience.
AVC and HEVC encoders on Snapdragon support Region of Interest (ROI) encoding to selectively improve image quality on focused subjects without compromising the overall bit-budget.
This provides the best of both worlds in terms of image quality and smaller file size. The ROI feature allows applications to specify multiple ROIs along with the QP Bias
values on specific frames indicated by the timestamps. Snapdragon Video encoders use ROI information to boost the relative quality within the specified regions
by spending more bits and selectively enhancing image quality while maintaining the overall bit-budget of the frame.
## [Visual Quality Improvement with ROI](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id10)[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#visual-quality-improvement-with-roi)
The following example shows a 1080p 30fps video recorded at a low bitrate of 3 Mbps. The left half of the image shows the compression artifacts on the subject and the background foliage.
The image on the right shows the same content encoded with the same configuration along with the ROI QP bias specified.
Quality difference on focused subject without and with ROI[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id1)
This image highlights the specified ROI rectangle to obtain the enhanced quality of the ROI enabled image above.
Specified ROI rectangle[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id2)
The following image shows close-up sections of the video that further emphasize visual quality differences in the ROI.
Detailed visual quality differences in regions of interest[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id3)
## [Enabling ROI Encoding on Snapdragon](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id11)[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#enabling-roi-encoding-on-snapdragon)
### [ROI extensions](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id12)[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#roi-extensions)
The following MediaCodec extensions are associated with the ROI encoding feature.
| API KEY | Value Type | Details |
| --- | --- | --- |
| [`KEY_ROI_RECT_INFO`](https://docs.qualcomm.com/doc/80-56386-10/topic/classqti_1_1video_1_1QMediaExtensions.html#_CPPv4N3qti5video16QMediaExtensionsE) | String | Payload contains lists of up to 5 rectangles with QP bias in format. |
| [`KEY_ROI_RECT_INFO_EXT`](https://docs.qualcomm.com/doc/80-56386-10/topic/classqti_1_1video_1_1QMediaExtensions.html#_CPPv4N3qti5video16QMediaExtensionsE) | String | Extension payload to configure additional rectangles. |
| [`KEY_ROI_RECT_TYPE`](https://docs.qualcomm.com/doc/80-56386-10/topic/classqti_1_1video_1_1QMediaExtensions.html#_CPPv4N3qti5video16QMediaExtensionsE) | String | Payload type. Currently only rectangles are supported.
This must be set to “rect”. |
| [`KEY_ROI_RECT_TIMESTAMP`](https://docs.qualcomm.com/doc/80-56386-10/topic/classqti_1_1video_1_1QMediaExtensions.html#_CPPv4N3qti5video16QMediaExtensionsE) | Integer | Timestamps are in microseconds. Must correspond to the
frame-timestamp of the frame attached to this payload. |
### [Configuring ROI-type](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id13)[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#configuring-roi-type)
The [`KEY_ROI_INFO_TYPE`](https://docs.qualcomm.com/doc/80-56386-10/topic/classqti_1_1video_1_1QMediaExtensions.html#_CPPv4N3qti5video16QMediaExtensionsE) property can be used to specify the ROI ‘type’. Currently, only the rectangle type of ROI is
supported and it should be set to “rect”. This extension parameter is only accepted at the time of configuration via MediaCodec.configure().
MediaCodec codec = MediaCodec.createEncoderByType("video/hevc");
MediaFormat format = MediaFormat.createVideoFormat("video/hevc", 1280, 720);
format.setString(KEY_ROI_INFO_TYPE, "rect");
...
codec.configure(format, ...);
Copy to clipboard
### [Configuring ROI Rectangles and QP Bias values](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id14)[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#configuring-roi-rectangles-and-qp-bias-values)
The [`KEY_ROI_RECT_INFO`](https://docs.qualcomm.com/doc/80-56386-10/topic/classqti_1_1video_1_1QMediaExtensions.html#_CPPv4N3qti5video16QMediaExtensionsE) property can be used to specify the ROI and the corresponding QP-bias values.
Applications can provide up to five overlapping rectangle regions.
The payload contains a list of up to five rectangles along with QP-bias in the format “top,left-bottom,right=QP”. Information for each region is separated by a semi-colon.
Negative QP bias values will result in a higher quality image as they supress the QP value. Positive QP bias values will reduce image quality.
Applications can use a combination of rectangles and QP bias values to specify emphasis area within a frame.
The following example shows a frame where the application intends to improve the quality of two regions with two different QP\_bias values.
ROI Regions[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id4)
The application can specify ROI regions with multiple rectangles to approximate the overall focus region.
In our example, the inner region is defined by rect(top=y5, left=x5 - bottom=y6, right=x6) and QP\_bias = -6 to showcase a stronger emphasis on quality.
The outer region is defined by two rects (top=y1, left=x1 - bottom=y2, right=x2) and (top=y3, left=x3 - bottom=y4, right=x4). Both regions have
QP\_bias = -3, showing a slightly lower emphasis than the inner region.
Rectangles Approximating the ROI[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id5)
The following code will set the ROI rectangles with QP-bias values for the regions shown in the diagram above.
// Following code is just before queueing the input buffer at time 'inputTimeUs'
Bundle roiInfo = new Bundle();
/*
* note: When there is an overlap, the QP-bias of the rectangle specified last
* ovewrites the QP bias of the rectangles specified before it.
*
* In this example, the inner rectangle with stronger bias is hence specified last.
*/
roiInfo.setString(KEY_ROI_RECT_INFO, "y1,x1-y2,x2=-3; y3,x3-y4,x4=-3; y5,x5-y6,x6=-6");
codec.setParameters(roiInfo);
/*
* queue the input buffer
*/
Copy to clipboard
Note
If the applications need to configure more than five regions, the extension [`KEY_ROI_RECT_INFO_EXT`](https://docs.qualcomm.com/doc/80-56386-10/topic/classqti_1_1video_1_1QMediaExtensions.html#_CPPv4N3qti5video16QMediaExtensionsE) must be used.
Once the ROI rectangles are received, the encoder will merge all regions in the supplied order.
Regions formed by rectangles covering the indended ROI[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id6)
Encoders compute QP at LCU ([Largest Coding Unit](https://en.wikipedia.org/wiki/Coding_tree_unit)) level, aligning the ROI rectangle to the nearest LCU boundary:
Regions stretched out to align to LCU boundaries[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id7)
The following example representation shows the QP values computed by the encoder and further offset by the QP-bias provided by the application.
Applying QP-bias offset in the encoder[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id8)
### [Configuring timestamps and queueing the input frame](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id15)[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#configuring-timestamps-and-queueing-the-input-frame)
The [`KEY_ROI_INFO_TIMESTAMP`](https://docs.qualcomm.com/doc/80-56386-10/topic/classqti_1_1video_1_1QMediaExtensions.html#_CPPv4N3qti5video16QMediaExtensionsE) parameter must be set to correlate the QP-bias information to the corresponding input frame.
The timestamp is in microseconds and must be same as the timestamp of the corresponding input frame.
// Following code is just before queueing the input buffer at time 'inputTimeUs'
Bundle roiInfo = new Bundle();
roiInfo.setString(KEY_ROI_RECT_INFO, "y1,x1-y2,x2=-3; y3,x3-y4,x4=-3; y5,x5-y6,x6=-6");
// applies the QP-bias regions at inputTimeUs
ROIRect.setInteger(KEY_ROI_INFO_TIMESTAMP, inputTimeUs);
codec.setParameters(roiInfo);
/*
* queue the input buffer
*/
if (usingByteBuffer) {
// queue the input buffer
codec.queueInputBuffer(id, ..., inputTimeUs, ...);
} else if (usingSurface) {
...
EGLExt.eglPresentationTimeANDROID(EGLDisplay, EGLSurface, inputTimeUs * 1000);
...
EGL14.eglSwapBuffers(EGLDisplay, EGLSurface);
...
} else if (usingImageWriter) {
...
// android.media.Image
image.setTimeStamp(inputTimeUs * 1000);
...
imageWriter.queueInputImage(image);
}
Copy to clipboard
### [Query capabilities](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#id16)[](https://docs.qualcomm.com/doc/80-56386-10/topic/roi.html#query-capabilities)
The application must check the ROI extensions in the vendor extensions list to confirm whether the ROI encoding feature is available on the device.
MediaCodec codec = MediaCodec.createByCodecName("c2.qti.hevc.encoder");
List vendorParams = codec.getSupportedVendorParameters();
/*
* vendorParams should contain KEY_ROI_RECT_INFO, KEY_ROI_RECT_INFO_EXT, KEY_ROI_INFO_TYPE
* and KEY_ROI_RECT_TIMESTAMP
*/
if (supportedExtensions.indexOf(KEY_ROI_INFO_TYPE) != -1
&& supportedExtensions.indexOf(KEY_ROI_RECT_INFO) != -1
&& supportedExtensions.indexOf(KEY_ROI_RECT_INFO_EXT) != -1
&& supportedExtensions.indexOf(KEY_ROI_RECT_TIMESTAMP) != -1) {
// ROI feature is supported
}
Copy to clipboard
Last Published: Jun 13, 2023
[Previous Topic
Extension Features](https://docs.qualcomm.com/bundle/publicresource/80-56386-10/topics/extension_features.md) [Next Topic
Encoder Initial QP Override](https://docs.qualcomm.com/bundle/publicresource/80-56386-10/topics/encoder_qp.md)