# AI USB camera
Source: [https://docs.qualcomm.com/doc/80-70022-50/topic/ai-usb-camera.html](https://docs.qualcomm.com/doc/80-70022-50/topic/ai-usb-camera.html)
The **gst-ai-usb-camera-app** streams video from a USB webcam connected to the
Qualcomm EVK. This webcam should be accessible as a /dev/videoX device.
Additionally, you can perform object detection and preview the results.
You can choose to preview the output on Wayland, or encode to a video file, or live
stream through the RTSP.
Alternatively, you can set `enable-object-detection `as *True* to
perform object detection.
The following figures show a pipeline, which processes the input from the USB camera to
generate various outputs.
For more information about the plugins used in these pipelines, see [Pipeline flow](https://docs.qualcomm.com/doc/80-70022-50/topic/ai-usb-camera.html#ai-usb-camera__section_jjq_bln_42c).
Figure : gst-ai-usb-single-camera-app pipeline
Live camera preview on display
v4l2src
qtivtransform
capsfilter waylandsink
Video encoding
v4l2src
qtivtransform
capsfilter v4l2h264enc
h264parse
mp4mux filesink
h264parse qtirtspbin Video encoding and RTSP streaming
v4l2src
qtivtransform
capsfilter v4l2h264enc
Qualcomm
Open source
Figure : gst-ai-usb-single-camera-app pipeline with object detection
v4l2src
qtimlvtransform
qtimlvconverter
qtimltflite/ qtimlsnpe/ qtimlqnn
qtimlpostprocess
qtivcomposer
waylandsink
capsfilter
tee
v4l2src
qtimlvtransform
qtimlvconverter
qtimltflite/ qtimlsnpe/ qtimlqnn
qtimlpostprocess
qtivcomposer
filesink
capsfilter
tee
v4l2src
qtimlvtransform
qtimlvconverter
qtimltflite/ qtimlsnpe/ qtimlqnn
qtimlpostprocess
qtivcomposer
RTSP sink
capsfilter
tee
Qualcomm
Open source
## Sample model and label files
| Runtime | Model files | Label files |
| --- | --- | --- |
| Qualcomm Neural Processing SDK | yolonas.dlc yolov5.dlc yolov8.dlc yolonas.json yolov8.json yolov5.json yolox.json yolov8_det_quantized.tflite yolonas_quantized.tflite yolov5.tflite yolox_quantized.tflite yolonas.json yolov8.json yolov5.json yolox.json yolonas.json yolov8.json yolov5.json yolox.json ",
"height": "",
"framerate": "",
"video-format": "",
"output": "",
"ip-address":"",
"port":"",
"enable-object-detection": "",
"file-path": "",
"yolo-model-type": "",
"model": "",
"labels": "",
"threshold": "",
"runtime": "",
"snpe-layers": ""
}Copy to clipboard
For example, run the application using RTSP input, LiteRT,
YOLOx model, DSP runtime, and custom threshold value. The reference config
file is as follows:
{
"width": 1920,
"height": 1080,
"framerate": 30,
"output":"waylandsink",
"video-format":"yuy2",
"model":"/etc/models/yolox_quantized.tflite",
"labels":"/etc/labels/yolox.json",
"output-file":"/etc/media/output.mp4",
"ip-address":"127.0.0.1",
"port":"8900",
"enable-object-detection": "TRUE",
"ml-framework": "tflite",
"yolo-model-type": "yolox",
"threshold": 75,
"runtime": "dsp"
}Copy to clipboard
4. Run the gst-ai-usb-camera-app
application:
gst-ai-usb-camera-app --config-file=/etc/configs/config-usb-camera-app.jsonCopy to clipboard
Note: You can run the
Yolo-NAS-Quantized.tflite model using the YOLOv8
value in the `yolo-model-type` field.
5. To display the available help options, run the following command in the SSH
shell:
gst-ai-usb-camera-app -hCopy to clipboard
6. To stop the use case, use CTRL +
C.
## Expected output
The following table lists the expected output for each use case:
Table : Expected output for gst-usb-single-camera-app
| Use case | Output |
| :--- | :--- |
| Preview | Preview of the camera stream. See [Figure : Expected output for gst-ai-usb-camera-app application–Preview](https://docs.qualcomm.com/doc/80-70022-50/topic/ai-usb-camera.html#ai-usb-camera__fig_ihh_tqq_5bc). |
| Save the MP4 | Save the output to a file atUSB camera captures the camera live stream. qtivtransform transforms the streamCapsfilter is applied to enforce constraints on the raw Filesink is used to dump the data into a file. USB camera captures the camera live stream. qtivtransform transforms the streamCapsfilter is applied to enforce constraints on the raw v4l2h264enc is used to encode the videoH264parse is used to parse the video. Mp4mux is used to multiplex the video into an MP4 Filesink is used to write the video to a file. USB camera captures the camera live stream. qtivtransform transforms the streamCapsfilter is applied to enforce constraints on the raw v4l2h264enc is used to encode the videoH264parse is used to parse the video. qtirtspbin is used to load the stream to RTSP. USB camera captures the camera live stream. Capsfilter is applied to enforce constraints on the raw tee is used to split the stream for inferencing. qtivtransform transforms the streamqtimlvconverter does preprocessing andqtimlsnpe , qtimltflite , or qtimlqnn run the inference on theqtimlvdetection handles the inferenceqtivcomposer composes the video framesqtirtspbin submits the composed videoUSB camera captures the camera live stream. Capsfilter is applied to enforce constraints on the raw tee is used to split the stream for inferencing. qtivtransform transforms the streamqtimlvconverter does preprocessing andqtimlsnpe , qtimltflite , or qtimlqnn run the inference on theqtimlpostprocess handles the inference results from any qtivcomposer composes the video framesWaylandsink submits the composed videoUSB camera captures the camera live stream. Capsfilter is applied to enforce constraints on the raw tee is used to split the stream for inferencing. qtivtransform transforms the streamqtimlvconverter does preprocessing andqtimlsnpe , qtimltflite , or qtimlqnn run the inference on theqtimlpostprocess handles the inference results from any qtivcomposer composes the video framesFilesink submits the composed video stream to Weston, which snpe–Qualcomm Neural Processing SDKtflite–LiteRTqnn–Qualcomm AI Engine directfilesinkwaylandsinkrtspsinkwidth: Input USB camera source resolutionheight: Input USB camera source resolutionframerate: Input USB camera source