# QIMP SDK release 1.2

Source: [https://docs.qualcomm.com/doc/80-70015-52/topic/qimp-sdk-release-1-2.html](https://docs.qualcomm.com/doc/80-70015-52/topic/qimp-sdk-release-1-2.html)

## Release information

Table : Software version

| Software | Version |
| --- | --- |
| Yocto | Kirkstone 4.0.20 |
| Kernel | 6.6.38 |

Table : Release tag version

| Release tag | Version |
| --- | --- |
| Firmware release tag | r1.0\_00049.0 |
| Release tag | qcom-6.6.38-QLI.1.2-Ver.1.1 |
| Meta-qcom-extras release tag | r1.0\_00050.0 |
| QIMP SDK release tag | qcom-6.6.38-QLI.1.2-Ver.1.1\_qim-product-sdk-1.1.2 |

Table : Supported platforms and reference kits

| SoC platforms | Reference kits |
| --- | --- |
| QCS6490 | <ul class="ul" id="qimp-sdk-release-1-2__ul_d11_1ff_vcc"><br>                                    <li class="li">Qualcomm<sup class="ph sup">®</sup> RB3 Gen 2 Vision Development Kit</li><br><br>                                    <li class="li">Qualcomm<sup class="ph sup">®</sup> RB3 Gen 2 Core Development Kit</li><br><br>                                </ul> |
| QCS5430 | <ul class="ul" id="qimp-sdk-release-1-2__ul_dsn_dff_vcc"><br>                                    <li class="li">Qualcomm<sup class="ph sup">®</sup> RB3 Gen 2 Lite Vision Development<br>                                        Kit</li><br><br>                                    <li class="li">Qualcomm<sup class="ph sup">®</sup> RB3 Gen 2 Lite Core Development<br>                                        Kit</li><br><br>                                </ul> |
| QCS9075 | Qualcomm^®^ IQ-9100 Beta Evaluation Kit |

## Contents of the release

The contents of the Qualcomm^®^ Intelligent Multimedia Product (QIMP) SDK
                release include:
- Recipes for building the individual components:
    - Qualcomm^®^ Intelligent Multimedia SDK (IM SDK)
    - TensorFlow Lite
    - Qualcomm^®^ Neural Processing SDK
    - Qualcomm^®^ AI Engine direct SDK
- Sample applications that demonstrate how to use the Qualcomm IM SDK to
                        develop AI edge-based applications.

To get started with the QIMP SDK, see [Qualcomm Intelligent Multimedia Product (QIMP)
                    SDK Quick Start Guide](https://docs.qualcomm.com/bundle/publicresource/topics/80-70015-51/introduction.html).

## New features

The following are the new features introduced in the QIMP SDK release:

- AI/ML supports:
    - Sample applications
        - `gst-ai-superresolution`, which allows high
                                    resolution video frames from low-resolution input.
        - `gst-ai-multistream-batch-inference`, which
                                    allows batched AI inference (object detection and segmentation)
                                    from a file source.
        - `gst-ai-multistream-inference`, which allows AI
                                    inference (object detection and classification) from multiple
                                    sources such as camera, file source, or RTSP.
        - `snpe_segmentation_app.py`, which allows
                                    segmentation using python bindings from a docker.
    - ⁠QuickSRNet for video super resolution.
    - Running batched models through the TensorFlow Lite plugin, enabling more
                            efficient processing of multiple inputs.
    - Enhanced workflow with stage-wise metadata, improving the clarity of the
                            daisy chain pipeline.
    - Message queuing telemetry transport (MQTT) communication through
                            dedicated plugins.
    - The Redissink plugin to submit ML metadata to the Redis database
                            server.
- Camera supports the following sample applications:
    - `gst-usb-single-camera-app`, which allows USB single
                            camera streaming for preview, video encoder, or network (RTSP).
    - `gst-camera-shdr-ldc-eis-example`, which allows super
                            high dynamic range (sHDR), low dynamic range (LDR), and electronic image
                            stabilization (EIS) camera imaging techniques.
    - `gst-smartcodec-example`, which reduces the network
                            bandwidth or storage from camera input.
- Video supports smart codec functionality, which dynamically adjusts bitrate,
                    framerate, and GOP based on the scene.
- Container supports:
    - Qualcomm IM SDK plugins and applications that allow you to use the same
                            Qualcomm IM SDK applications in both docker and nondocker
                            solutions.
    - Standalone TensorFlow Lite that lets you to have only TensorFlow Lite
                            and use their own pipeline for the rest of the application.
    - Standalone Snapdragon Neural Processing Engine (SNPE) for hardware
                            acceleration of AI models, enabling you to have only SNPE and use their
                            own pipeline for the rest of the application.
    - Python bindings on top of SNPE APIs and standalone Python applications
                            that support reading offline data, preprocessing, inferencing, and
                            postprocessing.
    - Standalone Qualcomm Neural Network (QNN) for hardware acceleration of AI
                            models, enabling you to have only SNPE and use their own pipeline for
                            the rest of the application.

## QCS9075 features

The following are the features introduced in the QIMP SDK release:
- AI/ML supports parallel inference for up to 24 streams.
- Video supports:
    - MJPEG video decoding and encoding sessions in CPU/GPU.
    - End‑to‑end 1080p concurrent video sessions for decode
                                and encode:
        - 24 H264/H265 sessions decode
        - 24 H264 sessions encode
        - 12 H264 sessions decode and encode
    - 4K at 240 FPS VP9 and AV1 decode
- Sample applications:

| Sample applications | Demonstrates... |
    | --- | --- |
    | **AI/ML** | **AI/ML** |
    | `gst-ai-classification` | Classification on streams from a camera, file source,<br>                                            or real-time streaming protocol (RTSP). |
    | `gst-ai-object-detection` | Object detection on streams from a camera, file<br>                                            source, or RTSP. |
    | `gst-ai-pose-detection` | Pose detection on streams from a camera, file source,<br>                                            or RTSP. |
    | `gst-ai-segmentation` | Image segmentation on streams from a camera, file<br>                                            source, or RTSP. |
    | `gst-ai-multi-input-output-object-detection` | Use cases for Edge AI boxes, which provide inputs<br>                                            through multiple sources such as camera, file source, or<br>                                            RTSP. |
    | `gst-ai-daisychain-detection-classification` | Cascaded object detection and classification on<br>                                            images streamed from multiple sources such as camera,<br>                                            file source, or RTSP. |
    | `gst-ai-daisychain-detection-pose` | Cascaded object detection and pose detection from<br>                                            multiple sources such as camera, file source, or<br>                                            RTSP. |
    | `gst-ai-superresolution` | High resolution video frames from low-resolution<br>                                            input. |
    | `gst-ai-multistream-inference` | AI inference (object detection and classification)<br>                                            from multiple sources such as camera, file source, or<br>                                            RTSP. |
    | `gst-ai-multistream-batch-inference` | Batched AI inference (object detection and<br>                                            segmentation) from file source. |
    | `snpe_segmentation_app.py` | Segmentation using python bindings from a<br>                                            docker. |
    | **Camera** | **Camera** |
    | `gst-smartcodec-example` | Reduce the network bandwidth or storage from camera<br>                                            input. |
    | **Video** | **Video** |
    | `gst-concurrent-videoplay-composition` | Playback multiple videos composed on a device. Use<br>                                            cases where video wall can be used in retail spaces and<br>                                            digital signage. |
    | `gst-videocodec-concurrent-playback` | Decoding and playback of multiple formats<br>                                            concurrently. Use cases for AI box and videoconferencing<br>                                            system, which sends video inputs in different<br>                                            formats. |
    | `gst-video-transcode-example` | Video transcoding of AVC-to-HEVC or HEVC-to-AVC<br>                                            format. |
    | `gst-transform-example` | Use cases where a video stream should be transformed<br>                                            (rotate, flip, and scale). |

## Sample applications

For a list of sample applications supported in the QIMP SDK and instructions on how
                to run them, see [Sample applications](https://docs.qualcomm.com/bundle/publicresource/topics/80-70015-50/example-applications.html).

## Python sample applications

- `gst-camera-detect-display`, which processes camera stream to
                    detect objects, overlay bounding boxes, and display the result.
- `gst-decode-detect-display`, which processes video stream to
                    detect objects and display them with bounding boxes.
- `gst-camera-two-stream-detection-and-classification-side-by-side`,
                    which detects and classifies objects from camera streams and displays the
                    results side by side.
- `gst-filesrc-2detection-classification-segmentation-side-by-side`,
                    which processes video files to detect, classify, and segment objects and
                    displays the results side by side.
- `gst-camera-two-stream-encode-file-detection-display`, which
                    encodes camera streams, detects objects, and displays the result.
- `gst-camera-three-stream-encode-file-detection-display-classification-rtsp`,
                    which encodes camera streams, displays detection results, and streams
                    classification results over RTSP.
- `gst-rtspsrc-detection-display`, which decodes an RTSP stream,
                    detects objects, overlays bounding boxes on them, and displays the result.
- `gst-camera-encode`, which records and encodes a camera
                    stream.
- `gst-camera-rotate-downscale-file`, which processes a single
                    camera stream by recording, downscaling, rotating, and encoding it.

## Limitations

The following are the known limitations in the QIMP SDK release:
- When using the qtioverlay plugin with detection models, frame drops may
                        occur, especially with many detections. 
    **Solution**: Use qtivcomposer
                            for detection-based ML use cases.
- A drop of 1-2 FPS may be observed with the three-stream camera use
                        case.
- AI/ML parallel inference for 24 streams is achieved at 22 FPS, which is
                        less than the expected 30 FPS.
- Segfault occurs when using Ctrl + C for SNPE use
                        cases.
- Frame drops are observed with the QNN plugin while running on GPU
                        delegate.
- Low FPS with daisychain detection and pose gstreamer pipeline.
- Gstreamer pipeline becomes unresponsive when you run 8 or more streams use
                        case with batched models.
- The `deeplabv3_resnet50.dlc` SNPE model gives 47 ms high
                        inference time.
- 18 FPS is observed with `gst-ai-parallel-inference` for
                        file source.
- `gst-ai-parallel-inference` hangs and black screen is
                        observed at EOS for RTSP source.
- RTSP sink streaming fails to play when you use multiple sinks such as RTSP,
                        file sink, and display with two or more input streams in
                            `multi-input-output-object-detection`.
- `gst-ai-monodepth` and
                            `gst-ai-parallel-inference` fails on QCS9075.

Last Published: Oct 15, 2024

[Next Topic
QIMP SDK release 1.1](https://docs.qualcomm.com/bundle/publicresource/80-70015-52/topics/ga1_1_release_information.md)