# QIMP SDK release 1.0 Source: [https://docs.qualcomm.com/doc/80-70014-52/topic/introduction.html](https://docs.qualcomm.com/doc/80-70014-52/topic/introduction.html) The Qualcomm^®^ Intelligent Multimedia Product Software Development Kit (QIMP SDK) is a comprehensive solution designed for application developers in the Internet of Things (IoT) sector. The QIMP SDK integrates the functionalities of the Qualcomm^®^ Intelligent Multimedia SDK (IM SDK), TensorFlow Lite SDK, Qualcomm^®^ Neural Processing SDK, and Qualcomm^®^ AI Engine direct SDK, facilitating the deployment of multimedia and artificial intelligence/machine learning (AI/ML) applications. The QIMP SDK uses GStreamer, an open-source multimedia framework, and offers user-friendly APIs and plugins in both multimedia and AI/ML domains. Key features of the QIMP SDK include: - APIs and tools for developing multimedia and AI applications. - The ability to create custom applications for multimedia development and AI acceleration using the TensorFlow Lite SDK, Qualcomm Neural Processing SDK, and Qualcomm AI Engine direct SDK. - Standalone compilation based on the Yocto Project. This release notes provide details about the contents of the release, new features, limitations, and instructions for building and flashing in this release. ## Release information Table : Software version | Software | Version | | --- | --- | | Yocto | kirkstone4.0.14 | | Kernel | 6.6.17 | Table : Release tag version | Release tag | Version | | --- | --- | | Firmware release tag | r1.0\_00026.0 | | Release tag | qcom-6.6.17-QLI.1.0-Ver.1.4 | | meta-qcom-extras release tag | r1.0\_00028.0 | | QIMP SDK release tag | qcom-6.6.17-QLI.1.0-Ver.1.4\_qim-product-sdk-1.1 | Table : Supported platforms | **SoC platform** | QCS6490 | | :--- | --- | ## Reference design kits Qualcomm^®^ Robotics RB3 Gen 2 Development Kit ## Contents of the release The contents of the QIMP SDK release include: - Recipes for building the individual components: - Qualcomm IM SDK - TensorFlow Lite - Qualcomm Neural Processing SDK - Qualcomm AI Engine direct SDK - Sample applications that demonstrate how to use the Qualcomm IM SDK to develop AI edge-based applications. ## New features - Camera supports: - Sample applications - gst-camera-single-stream-example, which allows single camera streaming to display or video encoder, or network (RTSP). - gst-multi-camera-example, which allows concurrent streaming from two cameras. - gst-multi-stream-example, which allows single camera data duplicated into two streams using [tee](https://gstreamer.freedesktop.org/documentation/coreelements/tee.html). - gst-snapshot-stream-example, which allows snapshot generation along with active video recording. - gst-activate-deactivate-streams-runtime-example, which allows camera resolution modifications on-the-fly without sensor restart. - gst-appsink-example, which is an application of transformations and other algorithms to a camera frame before encoding. - gst-camera-metadata-example, which is a metadata such as scene mode, white balance, and bit rate. - Real-time camera up to 4K at 30 FPS or 1080p at 60 FPS - Multistreams directly from a single camera - Multiple camera access from single or multiple client applications - Multiple snapshot support (RAW/JPEG) at configurable resolutions - Common camera controls like contrast, international organization for standardization (ISO), white balance, and so on using the GStreamer camera plugin properties - Camera metadata for fine-grain control of camera configurations - Video supports: - Sample applications - gst-audio-video-encode, which allows encoding of audio and video data, which is then stored into a user-specified output file. - gst-concurrent-videoplay-composition, which allows to play back multiple videos composed on a device. - gst-videocodec-concurrent-playback, which allows decoding and playback of multiple formats concurrently. - gst-audio-video-playback, which has audio and video playback capabilities. - gst-video-transcode-example, which allows video transcoding of AVC-to-HEVC or HEVC-to-AVC format. - gst-video-playback-example, which allows decoding and playback of a video file. - The file-based decode and playback feature - Video for Linux (V4L2)-based hardware encode and decode - 16 parallel instances for encode and decode - Encode supports: - Codecs: H.264, H.265 - Controls: Profile, level, bit rate control, minimum and maximum quantization parameter (QP), group of pictures (GOP), entropy coding, slice, Hier-P, rotation, flip, and long term reference (LTR) - Dynamic property control: LTR, bit rate, and flip - 4K30 + 720p30 encode - Decode supports: - Formats: NV12, TP10 universal bandwidth compression (UBWC) - Playback controls: Play, pause, resume, forward, and reverse - Dynamic resolution change - Video graphics array (VGA at 640 x 480) with 16 parallel instances - 720p with 8 parallel instances - 1080p with 4 parallel instances - UHD with 2 parallel instances - Audio supports: - Sample applications - gst-audio-encode-example, which allows audio recording. - gst-audio-decode-example, which allows playback of an audio file. - gst-transform-example, which has use cases where a video stream should be transformed (rotate, flip, and scale). - gst-weston-composition-example, which allows composition of various sources, specifically live camera input, and an offline file. - Hardware-accelerated audio capture and playback using pulse plugins - Graphics supports: - Hardware-accelerated plugins for compose, transform, and conversion - Composition: N input to 1 output. For example: video wall, ML metadata, and picture-in-picture - Transform: Crop, rotation, scale (down/up), color convert, and flip - Split: 1 input to N outputs based on user input region of interest (ROI) - AI/ML supports: - Sample applications - gst-ai-classification, which allows classification on a live camera stream. - gst-ai-multi-input-output-object-detection, which allows AI/ML inferencing on various input sources. - gst-ai-object-detection, which allows object detection on images streamed from a camera. - gst-ai-parallel-inference, which allows multiple AI/ML models running in parallel on a single live stream from a camera. - gst-ai-pose-detection, which allows pose detection on images streamed from a camera. - gst-ai-segmentation, which allows object segmentation on images streamed from a camera. - ML framework: TensorFlow Lite, Qualcomm Neural Processing SDK and Qualcomm AI Engine direct SDK - Models: Classification, detection, pose estimation, and segmentation - Tensor-based input and output for fast processing - Submodule-based postprocessing for flexibility of adding support for new models - Overlay inference results on input frame for complete ML use case realization - Miscellaneous includes: - Containerization reference - Socket source and sink plugins that: - Enable zero copy data path between two separate processes running their own GStreamer pipeline - Separate processes/applications are either within the native layer or within the docker execution environment ## Sample applications Sample applications are organized into two categories, each tailored to specific use cases like retail, AI edge box, IP camera, drones, and robotics. - Multimedia applications: These are centered around camera, video, audio, and graphics functionalities. - AI/ML applications: These primarily focus on AI and ML capabilities. For the list of sample applications supported in the QIMP SDK and instructions to run these applications, see [Sample applications](https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-50/example-applications.html). ## Limitations The following are known limitations in the QIMP SDK release: - The FastCV engine of qtimlvconverter does not support nonstandard resolution conversion. Use the OpenGL for embedded systems (GLES) engine as the convertor on this platform. - When using the qtioverlay plug-in with detection models, frame drops may occur, especially with many detections. Use qtivcomposer for detection-based ML use cases. - Speed-based video playback is not supported in this release. - A drop of 1-2 FPS may be observed with the three-stream camera use case. Last Published: Jul 13, 2024 [Previous Topic QIMP SDK release 1.1](https://docs.qualcomm.com/bundle/publicresource/80-70014-52/topics/ga1_1_release_information.md) [Next Topic Build and flash instructions](https://docs.qualcomm.com/bundle/publicresource/80-70014-52/topics/build_and_flash_instructions.md)