# LLM workflows with vLLM

These images are optimized for large language model inference using vLLM and
provide OpenAI-compatible APIs out of the box.

**Recommended images:**

- `cloud_ai_inference_vllm`
- `cloud_ai_inference_vllm_085`
- `cloud_ai_inference_vllm_py312` (for Python 3.12 / gpt-oss)

**Typical usage:**

- Start vLLM server directly
- Serve models using OpenAI-compatible endpoints
- Integrate with existing LLM clients

**Example: vLLM inference server**

docker pull ghcr.io/quic/cloud_ai_inference_vllm:1.21.2.0
    Copy to clipboard

docker run --rm -it \
      --shm-size=2gb \
      --network host \
      -e HF_TOKEN=<your_hf_token> \
      -v $PWD/hf_cache:/workspace/hf_cache \
      --device /dev/accel/ \
      ghcr.io/quic/cloud_ai_inference_vllm:1.21.2.0 \
      --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 \
      --port 8000
    Copy to clipboard

Last Published: Aug 25, 2026

[Previous Topic
Set up containers with Docker](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/index_Docker.md) [Next Topic
Disaggregated LLM Serving](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/disagg-workflows.md)