# Local LLMs

Run LLMs locally on Snapdragon X and X2 Series platforms using a choice of local models and model runtimes. Local LLMs bring AI directly onto the device, enabling private, low-latency, offline, and cost-efficient intelligence. They are a key enabler for AI PCs and edge computing because they allow you to run powerful AI workloads without relying on the cloud while keeping model execution and data on the device.

- Why local LLMs matter for AI PCs:
    - For platforms like Windows on Snapdragon, local LLMs can leverage:

> 
> 
> - NPU (Neural Processing Unit)
> - GPU
> - CPU

- Benefits:
    - - Longer battery life compared to cloud-heavy AI workloads
- Lower latency
- Improved privacy
- AI features that continue to work offline

- Examples:
    - - On-device summarization
- Document Q&A
- Code assistance
- Image generation
- Meeting note generation
- Personal knowledge assistants

The following section shows how to download and run an LLM locally using Ollama. Models are also available from the [Qualcomm AI Hub](https://aihub.qualcomm.com) and [Foundry Local](https://www.foundrylocal.ai/).

## Install Ollama and run a local LLM

[Ollama](https://ollama.com/) is a free, open-source project that lets you run open-source LLMs locally on Snapdragon X and X2 Series devices. It provides a simple API for creating, running, and managing models, as well as a library of [pre-built models](https://ollama.com/search) including the following:

- Llama 3.2 (Meta)
- Gemma 2 (Google)
- Phi 3.5 (Microsoft)
- Qwen 2.5 (Alibaba)
- Granite Code (IBM)
- Mistral (Mistral AI)
- Arctic Embed (Snowflake)

### Get started

Follow these steps to install Ollama, download a model to your Snapdragon device, and start an interactive chat session locally.

1. Select [OllamaSetup.exe](https://ollama.com/download/OllamaSetup.exe) to download the Ollama installer for Windows.
2. After it downloads, open OllamaSetup and follow the prompts to complete the installation.
3. After installation completes, open PowerShell.
4. Run the following command to start an interactive chat session with the `qwen2.5` model. If the model is not already available locally, Ollama downloads it first.

ollama run qwen2.5
        Copy to clipboard

    Ollama downloads the model if necessary and displays a prompt where you can enter your first message to start the chat session.

**Related**

- For more information about how to get started with Ollama, see [Ollama Github](https://github.com/ollama/ollama)
- To integrate Python projects with Ollama, see the [Ollama Python library](https://github.com/ollama/ollama-python/tree/main).
- Learn how Ollama simplifies inference with open source models on [Snapdragon X Series](https://www.qualcomm.com/developer/blog/2024/10/ollama-simplifies-inference-open-sources-models-snapdragon-x-series-devices) devices.

Last Published: Sep 23, 2026

[Previous Topic
MLC-LLM](https://docs.qualcomm.com/bundle/publicresource/80-62010-1/topics/gpu-mlc-llm-usage-guide.md) [Next Topic
AnythingLLM](https://docs.qualcomm.com/bundle/publicresource/80-62010-1/topics/run_anythingllm.md)