# AI USB camera
Source: [https://docs.qualcomm.com/doc/80-70018-50/topic/ai-usb-camera.html](https://docs.qualcomm.com/doc/80-70018-50/topic/ai-usb-camera.html)
The **gst-ai-usb-camera-app** streams the video from a USB webcam connected to the
Qualcomm EVK. This webcam should be accessible as a /dev/videoX device.
Additionally, you can perform object detection and preview the results.
You can choose to preview the output, save the video encoder output, save the camera YUV
to a file, live stream through RTSP, or perform object detection and preview the
results.
Note: The video encoding use case isn't supported in
the current release.
The figures show pipeline, which processes the input from the USB camera to generate
various outputs.
For more information about the plugins used in these pipelines, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70018-50/topic/ai-usb-camera.html#ai-usb-camera__section_jjq_bln_42c).
Figure : gst-ai-usb-single-camera-app pipeline
Figure : gst-ai-usb-single-camera-app pipeline with object detection
## Sample model and label files
| Runtime | Model files | Label files |
| --- | --- | --- |
| Qualcomm Neural Processing SDK | *yolonas.dlc* | yolonas.labels yolov8.labels yolov8_det_quantized.tflite yolonas_quantized.tflite yolonas.labels yolov8.labels yolonas.labels yolov8.labels root@:/etc/modelsCopy to clipboard
- Note that the [downloaded
script](https://docs.qualcomm.com/doc/80-70018-50/topic/download-model-and-label-files.html) downloads the sample video.mp4 video to
the /etc/media directory. If you are using a custom video,
then ensure that you push the video to /etc/media and
update the file path in the application config.JSON file.
- Connect the display to the device using the HDMI port. For instructions, see
[Set up HDMI display](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/samples.html).
- Enable the
display:
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1Copy to clipboard
If you face issues while enabling camera or display, see [Camera troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-17/troubleshooting.html) and [Display troubleshooting](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-18/debug.html).
- Ensure that the USB webcam is connected to the device.
Note: If the USB camera isn't detected on the device,
Download the required firmware. See [Download PCIe to USB controller
firmware](https://docs.qualcomm.com/bundle/publicresource/topics/80-70018-8/pcie.html#download-pcie-to-usb-controller-firmware).
## Run the application
The sample application uses the
/etc/configs/config-usb-camera-app.json file to read the
input parameters. Ensure that you use the config JSON file to run the
application.
To create your own config JSON file, use [config-usb-camera-app.json](https://git.codelinaro.org/clo/le/platform/vendor/qcom-opensource/gst-plugins-qti-oss/-/blob/imsdk.lnx.2.0.0.r2-rel/gst-sample-apps/gst-ai-usb-camera-app/config-usb-camera-app.json?ref_type=heads) as a
reference.
1. Enter the SSH shell and copy the YOLO-NAS label files to YOLOv8:
cp /etc/labels/yolonas.labels /etc/labels/yolov8.labelsCopy to clipboard
2. Use the following format of the
/etc/configs/config-usb-camera-app.json
file:
{
"width": "",
"height": "",
"framerate": "",
"format": "",
"output": "",
"ip-address":"",
"port":"",
"enable-object-detection": "",
"file-path": "",
"yolo-model-type": "",
"model": "",
"labels": "",
"constants": "",
"threshold": ,
"runtime": ""
}Copy to clipboard
Note: Update the config JSON file based
on the model, input stream, and other properties. For more information, see
[Config JSON field description](https://docs.qualcomm.com/doc/80-70018-50/topic/ai-usb-camera.html#ai-usb-camera__section_qjx_hqq_32c).
For example,
run the application using RTSP input, LiteRT, YOLOv8 model, DSP runtime,
custom constants, and custom threshold value.
In the
config-usb-camera-app.json file, you can replace
model-detection.tflite with
yolov8\_det\_quantized.tflite and
detection.labels with
yolov8.labels.
{
"width": 640,
"height": 480,
"framerate": 30,
"format": "YUY2",
"output": "PREVIEW",
"ip-address":"127.0.0.1",
"port":"8900",
"enable-object-detection": "TRUE",
"ml-framework": "tflite",
"yolo-model-type": "yolov8",
"model": "/etc/models/model-detection.tflite",
"labels": "/etc/labels/detection.labels",
"constants": "YOLOv8,q-offsets=<21.0, 0.0, 0.0>,q-scales=<3.093529462814331, 0.00390625, 1.0>;",
"threshold": 40,
"runtime": "dsp"
}Copy to clipboard
3. Run the gst-ai-usb-camera-app
application:
gst-ai-usb-camera-app --od-config-file=/etc/configs/config-usb-camera-app.jsonCopy to clipboard
Note: You can run the
Yolo-NAS-Quantized.tflite model using the YOLOv8 value in
the `yolo-model-type` field.
To display the available help options, run the following command in the SSH
shell:
gst-ai-usb-camera-app -hCopy to clipboard
To stop the use case, use CTRL + C.
## Expected output
The table lists the expected output for each use case:
Table : Expected output for gst-usb-single-camera-app
| Use case | Output |
| :--- | :--- |
| Preview | Preview of the camera stream. See [Figure : Expected output for gst-ai-usb-camera-app application–Preview](https://docs.qualcomm.com/doc/80-70018-50/topic/ai-usb-camera.html#ai-usb-camera__fig_ihh_tqq_5bc). |
| Save the MP4 and YUV data | Save the output to a file at /etc/ folderUSB camera captures the camera live stream. qtivtransform transforms the streamCapsfilter is applied to enforce constraints on the raw Filesink is used to dump the YUV data into a file. USB camera captures the camera live stream. qtivtransform transforms the streamCapsfilter is applied to enforce constraints on the raw The data is sent to the Wayland display sink (Waylandsink ) for live preview. USB camera captures the camera live stream. qtivtransform transforms the streamCapsfilter is applied to enforce constraints on the raw v4l2h264enc is used to encode the videoH264parse is used to parse the video. Mp4mux is used to multiplex the video into an MP4 Filesink is used to write the video to a file. USB camera captures the camera live stream. qtivtransform transforms the streamCapsfilter is applied to enforce constraints on the raw v4l2h264enc is used to encode the videoH264parse is used to parse the video. qtirtspbin is used to load the stream to RTSP. USB camera captures the camera live stream. Capsfilter is applied to enforce constraints on the raw tee is used to split the stream for inferencing. qtivtransform transforms the streamqtimlvconverter does preprocessing andqtimlsnpe , qtimltflite , or qtimlqnn run the inference on theqtimlvdetection handles the inferenceqtivcomposer composes the video framesWaylandsink submits the composed videosnpe–Qualcomm Neural Processing SDKtflite–LiteRTqnn–Qualcomm AI Engine directcamera–Primary (0) or secondary (1).file-path–The directory path to the videortsp-ip-port–The address of the RTSPrtsp://<ip>:<port>/<stream>