# Camera Frame Access
Camera frame access allows developers to retrieve raw camera data in real time.
## How to enable the feature
In order to use Camera Frame Access, enable the OpenXR plugin settings located under **Project Settings > XR Plug-in Management > OpenXR (> Android Tab) > OpenXR Feature Groups > Snapdragon Spaces > Camera Access**.
Note
This feature is not currently available for Android XR devices!
## Feature Settings
Camera Frame Access has 3 feature settings:
- **Direct Memory Access Conversion**: Used by the application to decide how to access camera memory before CPU frame conversion. May improve performance in some devices, while degrading performance in others.
- **CPU Frame Cache Size**: Size of the XRCpuImage.ConvertAsync frame cache, defining how many asynchronous conversion requests can be queued. When the limit is reached, older requests expire.
- **Cache Frame Before Async Conversion**: Guarantees async conversions by caching the frames upon request without relying on the CPU Frame Cache, with a slight performance and memory penalty. Ignored if *Direct Memory Access Conversion* is enabled.
## Retrieving CPU images
Each time a new camera frame is available to the subsystem, the **AR Camera Manager** fires a `frameReceived` event. Subscribing to this event allows other scripts to retrieve the latest camera frame at the earliest time possible and perform operations on it. Once a camera frame is available, the camera manager’s `TryAcquireLatestCpuImage` function returns an `XRCpuImage` object which represents a single, raw image from the selected device camera. The raw pixel data of this image can be extracted with `XRCpuImage`’s `Convert` function, which returns a `NativeArray`.
Snapdragon Spaces also supports asynchronous conversion of frames through `XRCpuImage.ConvertAsync`, either by polling the returned `AsyncConversion` object and getting the data with `AsyncConversion.GetData`, or by providing a callback with a `NativeArray` argument to be executed when conversion is completed.
Tip
An **AR Camera Manager** component is automatically added to the scene hierarchy while creating an **XR Origin**. Camera frames are requested from the OpenXR Runtime application when an **AR Camera Manager** component is enabled. Consider disabling this component if you do not require Camera Frame Access in your scene.
Important
An **AR Camera Background** component is automatically added to the scene hierarchy when creating an **XR Origin**. This component must be disabled to avoid rendering issues mentioned in the [known issues section](../../releasenotes/content/knownissues#ar-camera-manager-and-ar-camera-background-issues).
Important
`XRCpuImage` objects must be explicitly disposed off after conversion. To do this, use `XRCpuImage`’s `Dispose` function. Failing to dispose the `XRCpuImage` objects may leak memory until the camera access subsystem is destroyed.
If allocating a `NativeArray` before conversion, this buffer must also be disposed after copy or manipulation. To do this, use `NativeArray`’s `Dispose` function. Failing to dispose the `NativeArray` leaks memory until the camera access subsystem is destroyed.
Important
`AsyncConversion` objects must be explicitly disposed of after the data has been used. To do this, use `AsyncConversion`’s `Dispose` function. Failing to dispose the `AsyncConversion` objects leaks memory until the camera access subsystem is destroyed.
When using `XRCpuImage.ConvertAsync` and providing a callback function, it is not necessary to dispose the the provided `NativeArray` data buffer. Conversion data is disposed by the camera subsystem when the callback finishes.
For detailed information about how to use `frameReceived`, `TryAcquireLatestCpuImage`, `XRCpuImage.Convert` and `XRCpuImage.ConvertAsync`, see the [Unity documentation](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.1/manual/features/Camera/image-capture.html).
Caution
Camera Frame Access may need a few seconds to initialize depending on the used device. Do not try to access frames before the subsystem has initialized successfully. It is highly recommended to use the `frameReceived` event from the AR Foundation API to avoid errors.
### Sample Code
Requests a CPU image from the **AR Camera Manager** when the `frameReceived` event is fired. If successful, it extracts the `XRCpuImage`’s raw pixel data directly into a managed `Texture2D`’s `GetRawTextureData` buffer, applying the texture buffer later, with the `Apply` function. Finally, it updates the texture in the target `RawImage`, making the new frame visible in the application’s UI.
public RawImage CameraRawImage;
private ARCameraManager _cameraManager;
private Texture2D _cameraTexture;
private XRCpuImage _lastCpuImage;
public void Start()
{
_cameraManager.frameReceived += OnFrameReceived;
}
private void OnFrameReceived(ARCameraFrameEventArgs args)
{
_lastCpuImage = new XRCpuImage();
if (!_cameraManager.TryAcquireLatestCpuImage(out _lastCpuImage))
{
return;
}
UpdateCameraTexture(_lastCpuImage);
}
private unsafe void UpdateCameraTexture(XRCpuImage image)
{
var format = TextureFormat.RGBA32;
if (_cameraTexture == null || _cameraTexture.width != image.width || _cameraTexture.height != image.height)
{
_cameraTexture = new Texture2D(image.width, image.height, format, false);
}
var conversionParams = new XRCpuImage.ConversionParams(image, format);
var rawTextureData = _cameraTexture.GetRawTextureData();
try
{
image.Convert(conversionParams, new IntPtr(rawTextureData.GetUnsafePtr()), rawTextureData.Length);
}
finally
{
image.Dispose();
}
_cameraTexture.Apply();
CameraRawImage.texture = _cameraTexture;
}
Copy to clipboard
The following texture formats are supported by the **AR Camera Manager**:
- `RGB24`
- `RGBA32`
- `BGRA32`
## Retrieving Camera Frames as Texture2D
Note
This feature is available only for compatible Snapdragon Spaces all-in-one devices.
### Sample Code
Requests a Texture2D image from the `ARCameraFrameEventArgs` variable when the `frameReceived` event is fired. If the textures array inside `args` is not empty, it means that there are RGB textures ready for its use. These textures have been converted through the GPU using shaders. This results in a faster conversion from YUV to RGB and therefore, improving the performance.
public RawImage CameraRawImage;
private ARCameraManager _cameraManager;
public void Start()
{
_cameraManager.frameReceived += OnFrameReceived;
}
private void OnFrameReceived(ARCameraFrameEventArgs args)
{
CameraRawImage.texture = args.textures.Count > 0 ? args.textures[0] : null;
}
Copy to clipboard
## Retrieving YUV plane data
Snapdragon Spaces currently supports the `Y'UV420sp` and `YUY2` formats. `Y'UV420sp` consists of a Y buffer followed by an interleaved 2x2 subsampled U/V buffer. `YUY2` consists of a single buffer interleaving Y and U/V samples in the form of Y-U-Y-V ‘macropixels’; each sample represents 2 horizontal pixels with equal chrominance, with no vertical subsampling. For detailed information about the YUV color model, see the [YUV section of the YCbCr Wikipedia article](https://en.wikipedia.org/wiki/YCbCr#Packed_pixel_formats_and_conversion).
If RGB conversion is not needed, the raw YUV plane data can be retrieved through `XRCpuImage`’s `GetPlane` function. This returns an `XRCpuImage.Plane` object from which plane data can be read. Formats can be differentiated with `XRCpuImage.planeCount`, indicating the amount of planes representing the frame.
**Y’UV420sp**
- **Y plane** data has index **0** and can be accessed via `GetPlane(0)`
- **UV plane** data has index **1** and can be accessed via `GetPlane(1)`
**YUY2**
- **YUYV plane** data has index **0** and can be accessed via `GetPlane(0)`
Warning
In case multiple applications are accessing a sensor at the same time, they may see a `YUY2` frame format where they normally would see a `Y'UV420sp` frame format. It is recommended to handle both cases in your application. Not doing so might cause issues when using video capture or streaming services.
For detailed information about `XRCpuImage.GetPlane`, see the [Unity documentation](https://docs.unity3d.com/Packages/com.unity.xr.arsubsystems@4.2/api/UnityEngine.XR.ARSubsystems.XRCpuImage.html#UnityEngine_XR_ARSubsystems_XRCpuImage_GetPlane_System_Int32_).
For detailed information about `XRCpuImage.Plane`, see the [Unity documentation](https://docs.unity3d.com/Packages/com.unity.xr.arsubsystems@4.2/api/UnityEngine.XR.ARSubsystems.XRCpuImage.Plane.html).
### Sample Code
Requests a CPU image from the **AR Camera Manager** when the `frameReceived` event is fired. If successful, it retrieves the `XRCpuImage`’s raw plane data to apply different image processing algorithms to it depending on the YUV format.
private ARCameraManager _cameraManager;
private XRCpuImage _lastCpuImage;
public void Start()
{
_cameraManager.frameReceived += OnFrameReceived;
}
private void OnFrameReceived(ARCameraFrameEventArgs args)
{
_lastCpuImage = new XRCpuImage();
if (!_cameraManager.TryAcquireLatestCpuImage(out _lastCpuImage))
{
return;
}
switch(_lastCpuImage.planeCount)
{
case 1:
ProcessYuy2Image(_lastCpuImage);
break;
case 2:
ProcessYuv420Image(_lastCpuImage);
break;
}
}
private void ProcessYuv420Image(XRCpuImage image)
{
var yPlane = image.GetPlane(0);
var uvPlane = image.GetPlane(1);
for (int row = 0; row < image.height; row++)
{
for (int col = 0; col < image.width; col++)
{
// Perform image processing here...
}
}
}
Copy to clipboard
## Retrieving Sensor Extrinsics
The AR Foundation API does not expose sensor extrinsics. Instead, two methods are provided to get the sensor extrinsics.
Snapdragon Spaces provides input actions for **ColorCameraPosition** and **ColorCameraRotation**. These input actions can be used with a **Tracked Pose Driver (Input System)** by binding them with the **Position Input** and **Rotation Input**. This **Tracked Pose Driver (Input System)** can be added to a GameObject and the pose of this GameObject matches the color camera extrinsics pose.
Tip
As the **Tracked Pose Driver (Input System)** component relies on the update timing of the underlying **Input System**, the transform of the associated GameObject might be outdated by one frame when requested.
If minimal latency is required or if your system has low error tolerance, position and rotation can instead be retrieved with `TrackedInputDriver.positionAction.ReadValue()` and `TrackedInputDriver.rotationAction.ReadValue()` respectively.
The following image shows how to use these input bindings.
Add the **Spaces Camera Pose Provider** component to the scene and use the `GetPoseFromProvider` function to retrieve the camera pose associated with the **AR Camera Manager**’s latest `frameReceived` event. The provided pose is relative to the **XR Origin**’s coordinate system.
### Sample Code
Retrieves the extrinsics of the selected sensor and displays it in the application UI.
public SpacesCameraPoseProvider PoseProvider;
public Text[] ExtrinsicPositionTexts;
public Text[] ExtrinsicOrientationTexts;
private Pose _extrinsics;
private void UpdateCameraExtrinsics()
{
if (PoseProvider.GetPoseFromProvider(out _extrinsics) == PoseDataFlags.NoData)
{
Debug.LogWarning("Failed to acquire camera extrinsics.");
return;
}
var position = _extrinsics.position;
var orientation = _extrinsics.rotation.eulerAngles;
ExtrinsicPositionTexts[0].text = position.x.ToString();
ExtrinsicPositionTexts[1].text = position.y.ToString();
ExtrinsicPositionTexts[2].text = position.z.ToString();
ExtrinsicOrientationTexts[0].text = orientation.x.ToString();
ExtrinsicOrientationTexts[1].text = orientation.y.ToString();
ExtrinsicOrientationTexts[2].text = orientation.z.ToString();
}
Copy to clipboard
**Spaces Camera Pose Provider** can also be used as a pose provider for a **Tracked Pose Driver**.
## Retrieving Sensor Intrinsics
The camera manager’s `TryGetIntrinsics` function returns an `XRCameraIntrinsics` object which describes the physical characteristics of the selected sensor. For detailed information about `XRCameraIntrinsics`, please refer to the [Unity documentation](https://docs.unity3d.com/Packages/com.unity.xr.arsubsystems@4.2/api/UnityEngine.XR.ARSubsystems.XRCameraIntrinsics.html).
### Sample Code
Retrieves the intrinsics of the selected sensor and displays it in the application UI.
public Text[] ResolutionTexts;
public Text[] FocalLengthTexts;
public Text[] PrincipalPointTexts;
private ARCameraManager _cameraManager;
private XRCameraIntrinsics _intrinsics;
private void UpdateCameraIntrinsics()
{
if (!_cameraManager.TryGetIntrinsics(out _intrinsics))
{
Debug.LogWarning("Failed to acquire camera intrinsics.");
return;
}
ResolutionTexts[0].text = _intrinsics.resolution.x.ToString();
ResolutionTexts[1].text = _intrinsics.resolution.y.ToString();
FocalLengthTexts[0].text = _intrinsics.focalLength.x.ToString("#0.00");
FocalLengthTexts[1].text = _intrinsics.focalLength.y.ToString("#0.00");
PrincipalPointTexts[0].text = _intrinsics.principalPoint.x.ToString("#0.00");
PrincipalPointTexts[1].text = _intrinsics.principalPoint.y.ToString("#0.00");
}
Copy to clipboard
## Tips to increase performance
The Camera Access operations can be computationally expensive, either due to a large image resolution or due to the nature of the algorithms used.
- When converting the image with `XRCpuImage.Convert`, provide `XRCpuImage.ConversionParams` with smaller `outputDimensions`.
- When processing the image with `XRCpuImage.GetPlane`, consider subsampling the data buffers with a common factor.
- Consider processing the image with `XRCpuImage.ConvertAsync`, which avoids blocking the main application thread during conversion.
Snapdragon Spaces offers a **Direct Memory Access Conversion** setting, which changes the way `XRCpuImage.Convert` and `XRCpuImage.ConvertAsync` read and write data. By default, Snapdragon Spaces moves frame data using `Marshal.Copy`. Enabling this setting allows Spaces to use `NativeArray` direct representations of the source and target buffers instead. This setting can be found in `Project Settings > XR Plug-in Management > OpenXR (> Android Tab) > Snapdragon Spaces > Camera Frame Access > Direct Memory Access Conversion`. Enabling this setting may improve performance on certain devices, though it may negatively impact performance on other architectures.
## Retrieving Additional Sensor Data
The **Spaces AR Camera Manager Config** exposes the number of camera sensors in the device using the `GetActiveCameraCount` method. This information could be relevant for differentiating between dual devices and VR devices, and for appropriately handling the camera image.
Last Published: Jun 06, 2025
[Previous Topic
Spatial Anchors](https://docs.qualcomm.com/bundle/publicresource/80-88642-11/topics/features_anchors.md) [Next Topic
Composition Layers](https://docs.qualcomm.com/bundle/publicresource/80-88642-11/topics/features_compositionlayers.md)