# Linux and Windows ARM64 Genie Service Setup

## Purpose

This document defines processes to package, stage, start, validate, and troubleshoot Linux ARM64 and Windows ARM64 Genie Service deployments across supported devices.

## Required Inputs

- <DEVICE\_USER>, <DEVICE\_HOST>: target SSH credentials

## Supported Devices

| Platform | SoC | SoC ID | Architecture | DSP Version |
| --- | --- | --- | --- | --- |
| LeMans IoT | QCS9075 | 676 | aarch64-oe-linux-gcc11.2 | V73 |
| Monza IoT | QCS8275 | 675 | aarch64-oe-linux-gcc11.2 | V75 |
| Glymur | SC8480X | 662 | aarch64-windows-msvc | V81 |

## Installing Genie Service for Linux IoT

### Step 1: Build Genie Service Test Package

1. Create test package directory

> 
> 
> mkdir -p genie_service/lib genie_service/config genie_service/dsp
>         Copy to clipboard
2. Copy binary from Genie release:

> 
> 
> cp <release>/bin/aarch64-oe-linux-gcc11.2/genie-service genie_service/genie-service
>         Copy to clipboard
3. Copy Genie library dependencies from Genie release:

> 
> 
> cp <release>/lib/aarch64-oe-linux-gcc11.2/libGenie.so <release>/lib/aarch64-oe-linux-gcc11.2/libGenieService.so <release>/lib/aarch64-oe-linux-gcc11.2/libQnnHtp.so <release>/lib/aarch64-oe-linux-gcc11.2/libQnnHtpNetRunExtensions.so <release>/lib/aarch64-oe-linux-gcc11.2/libQnnHtp${DSP_VER}Stub.so <release>/lib/aarch64-oe-linux-gcc11.2/libQnnSystem.so genie_service/lib/
>         Copy to clipboard
4. Copy DSP library dependencies from Genie release:

> 
> 
> cp <release>/lib/hexagon-${DSP_VER}/lib/libQnnHtp${DSP_VER}Skel.so genie_service/dsp
>         Copy to clipboard
5. Copy Service Config:

> 
> 
> cp <release>/examples/Genie/OpenAI/config/genie_openai_service_config.yaml genie_service/config/
>         cp <release>/examples/Genie/OpenAI/config/genie_service.service genie_service/config/
>         Copy to clipboard
> 
>     - Note: Service config can be modified to use preferred models
6. Finalize:

> 
> 
> chmod +x genie_service/genie-service
>         chmod 755 genie_service/lib/*.so genie_service/dsp/*.so
>         tar -czf genie_service.tgz -C genie_service .
>         Copy to clipboard

### Step 2: Stage Package and Models on Device

1. Copy tarball to device

> 
> 
> scp genie_service.tgz <DEVICE_USER>@<DEVICE_HOST>:/tmp/genie_service.tgz
>         Copy to clipboard
2. Unzip tarball on device

> 
> 
> ssh <DEVICE_USER>@<DEVICE_HOST> 'rm -rf /tmp/genie_service && mkdir -p /tmp/genie_service && tar -xzf /tmp/genie_service.tgz -C /tmp/genie_service'
>         Copy to clipboard
3. Create model folder

> 
> 
> ssh <DEVICE_USER>@<DEVICE_HOST> 'mkdir -p ${TARGET_INSTALL_PATH}/models'
>         Copy to clipboard
4. Copy model files to device

> 
> 
> rsync -av <model_dir>/ <DEVICE_USER>@<DEVICE_HOST>:${TARGET_INSTALL_PATH}/models/<model_dir>/
>         Copy to clipboard
> 
>     - Note: Must copy Genie compatible models, including JSON configs and tokenizer JSON.
>     - Note: Ensure that the paths in the model config files are valid paths on device, ex. /tmp instead of /data/local/tmp.

### Step 3: Clear Stale Services

1. Before every run, take care to stop old genie-service listeners on chosen HTTP port.

> 
> 
> ssh <DEVICE_USER>@<DEVICE_HOST> 'pkill -x genie-service || true; for pid in $(pgrep -f "genie-service --config" || true); do kill "$pid" || true; done; sleep 2; pgrep -af "genie-service " || true; ss -ltnp 2>/dev/null | grep 18080 || true'
>         Copy to clipboard
> 
>     - Note: Multiple listeners listening on the same port can route requests to the wrong runtime.

### Step 4: Start New Service

1. On device:

> 
> 
> cd /tmp/genie_service
>         LD_LIBRARY_PATH=/tmp/genie_service/lib ADSP_LIBRARY_PATH=/tmp/genie_service/dsp /tmp/genie_service/genie-service --config /tmp/genie_service/config/genie_openai_service_config.yaml > /tmp/genie_service/service.log 2>&1 < /dev/null &
>         Copy to clipboard
> 
>     - Verify process, listening port, and recent logs.

### Step 5: Execute Python Test App

1. Update model IDs per test case in `OpenAIClientApp_test_suite.json` to match those configured on the device.

> 
> 
> - Note: Do not edit `OpenAIClientApp.py`.
2. From `<release>` directory, run:

> 
> 
> env NO_PROXY=<DEVICE_HOST>,127.0.0.1,localhost OCA_BASE_URL=http://<DEVICE_HOST>:18080 OCA_SKIP_MULTIMODAL=true OCA_SKIP_ADMIN=true python3 <release>/examples/Genie/OpenAI/OpenAIClientApp/OpenAIClientApp.py 2>&1 | tee <RESULTS_DIR>/<RUN_NAME>_python_test.log
>         Copy to clipboard
> 
>     - Exit code `0` means all tests passed or were skipped; exit code `1` means one or more tests failed or the service was unreachable.

## Installing Genie Service for Windows-on-ARM

### Prerequisites

- Shell: PowerShell 7+ (`pwsh.exe`)
- Elevation: Administrator required. A non-elevated shell refuses scripts with a `#requires` administrator statement.
- Execution policy: Set for the current session with `Set-ExecutionPolicy -Scope Process Bypass -Force`.
- Target platform: Windows-on-ARM (Snapdragon). The binaries are `aarch64` only.

### Getting an Elevated PowerShell

Any one of these methods works:

1. Start menu -&gt; search `PowerShell` -&gt; right-click -&gt; `Run as administrator`.
2. Windows Terminal -&gt; click dropdown -&gt; hold `Ctrl+Shift` and click `Windows PowerShell`.
3. From a non-admin PowerShell: `Start-Process powershell -Verb RunAs`.

Verify elevation:

([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
    Copy to clipboard

Will print `True` if administrator permissions are active

### Step 1: Generate Windows Genie Service Test Package

1. Locate QAIRT release and navigate to `<release>/examples/Genie/OpenAI/platform/windows`.
2. Run the package generation script:

> 
> 
> .\package_service.ps1 -Version '<VERSION>' -OutputDir '<OUTPUT_DIR>'
>         # ex: .\package_service.ps1 -Version '2.50' -OutputDir 'C:\Deploy'
>         Copy to clipboard

### Step 2: Push Windows Package and Models to Remote Device

From local machine, run the following commands to deploy local service and models to the remote device:

1. Copy tarball to remote device

> 
> 
> scp GenieService-<VERSION>-aarch64-windows.zip <DEVICE_USER>@<DEVICE_HOST>:C:\Deploy\GenieService-<VERSION>-aarch64-windows.zip
>         Copy to clipboard
2. Create model folder

> 
> 
> ssh <DEVICE_USER>@<DEVICE_HOST> mkdir C:/ProgramData/Genie/models
>         Copy to clipboard
3. Copy model files to device

> 
> 
> scp -r <model_dir>/ <DEVICE_USER>@<DEVICE_HOST>:C:/ProgramData/Genie/models
>         Copy to clipboard
> 
>     - Note: Must copy Genie compatible models, including JSON configs and tokenizer JSON.
>     - Note: Ensure that the paths in the model config files are valid paths on device pointing to correct files.
>     - Note: Ensure that the paths in the Genie Service config JSON are pointing to the correct model config file paths.

### Step 3: Extract the ZIP

On the remote device run the following commands to unzip the release:

> 
> 
> $ZipPath = 'C:\Deploy\GenieService-<VERSION>-aarch64-windows.zip'
>     $DestDir = 'C:\Deploy'
>     
>     Expand-Archive -Path $ZipPath -DestinationPath $DestDir -Force
>     Copy to clipboard

- Extracted Genie Service files will be located at: `C:\Deploy\GenieService-<VERSION>-aarch64-windows\`

### Step 4: Unblock Files

On the remote device run the following commands to unblock the files:

Get-ChildItem -Recurse 'C:\Deploy\GenieService-<VERSION>-aarch64-windows' | Unblock-File
    Copy to clipboard

### Step 5: Install and Start

On the remote device, run the following commands to install and start the service:

1. Allow scripts to run in this session.

> 
> 
> Set-ExecutionPolicy -Scope Process Bypass -Force
>         Copy to clipboard
2. Change into the extracted package directory.

> 
> 
> Set-Location 'C:\Deploy\GenieService-<VERSION>-aarch64-windows'
>         Copy to clipboard
3. Install and start (model files must already be staged; see genie\_openai\_service\_config.windows.json).

> 
> 
> .\deploy-genie-service.ps1 -ServiceRoot .\bin -RuntimeDir .\runtime -Config .\config\genie_openai_service_config.windows.json -Start
>         Copy to clipboard

- Expected output on success includes `CreateService SUCCESS`, `ChangeServiceConfig2 SUCCESS`, and service status `Running`.
- If the health poll fails and service reports `Stopped` after 5 seconds, check Application event log and service log destination.
- Note: The Genie Service can also be explicitly started and run in the console:

> 
> 
> & 'C:\Program Files\GenieOpenAIService\genie-service.exe' --console --config 'C:\Program Files\GenieOpenAIService\genie_openai_service_config.windows.json
>         Copy to clipboard

### Step 6: Verify the Install

On the remote device, run the following commands to verify the service installation:

1. Service is registered, boot-start, LocalSystem:

> 
> 
> Get-Service GenieService | Format-List Name, Status, StartType, ServiceType
>         sc.exe qc GenieService
>         sc.exe qfailure GenieService
>         Copy to clipboard
2. Files landed under the install dir:

> 
> 
> Get-ChildItem 'C:\Program Files\GenieOpenAIService'
>         Copy to clipboard
3. Inbound firewall rule:

> 
> 
> Get-NetFirewallRule -DisplayName 'GenieService (Inbound)' | Format-List DisplayName, Enabled, Direction, Action, Profile
>         Copy to clipboard
4. HTTP endpoint listening on default port 8080:

> 
> 
> Test-NetConnection -ComputerName localhost -Port 8080
>         Copy to clipboard
5. Optional: query the models endpoint.

> 
> 
> Invoke-RestMethod http://localhost:8080/v1/models
>         Copy to clipboard

### Step 7: Execute Python Test App

On the local machine, follow these steps to run the python test app to validate the service:

1. Unzip the release on a local machine to access the OpenAIClientApp.py test script. Navigate to the unzipped directory.
2. Update model IDs per test case in `examples/Genie/OpenAI/OpenAIClientApp/OpenAIClientApp_test_suite.json` to match those configured on the device.

> 
> 
> - Note: Do not edit `OpenAIClientApp.py`.
3. On a Linux local machine, from the top level directory of the unzipped release run:

> 
> 
> env NO_PROXY=<DEVICE_HOST>,127.0.0.1,localhost OCA_BASE_URL=http://<DEVICE_HOST>:8080 OCA_SKIP_MULTIMODAL=true OCA_SKIP_ADMIN=true python3 ./examples/Genie/OpenAI/OpenAIClientApp/OpenAIClientApp.py 2>&1 | tee <RESULTS_DIR>/<RUN_NAME>_python_test.log
>         Copy to clipboard
4. On a Windows local machine, from the top level directory of the unzipped release run:

> 
> 
> $env:NO_PROXY="<DEVICE_HOST>,127.0.0.1,localhost"; $env:OCA_BASE_URL="http://<DEVICE_HOST>:8080"; $env:OCA_SKIP_MULTIMODAL="true"; $env:OCA_SKIP_ADMIN="true"; python .\examples\Genie\OpenAI\OpenAIClientApp\OpenAIClientApp.py 2>&1 | Tee-Object -FilePath "<RESULTS_DIR>\<RUN_NAME>_python_test.log"
>         Copy to clipboard
> 
>     - Exit code `0` means all tests passed or were skipped; exit code `1` means one or more tests failed or the service was unreachable.

### To Uninstall Windows Service:

On the remote device, run the following commands to uninstall the service:

# From any elevated PowerShell; only needs package dir to find deploy-genie-service.ps1.
    Set-Location 'C:\Deploy\GenieService-<VERSION>-aarch64-windows'
    .\deploy-genie-service.ps1 -Uninstall
    Copy to clipboard

- This stops the service, deletes the SCM entry, removes the inbound firewall rule, and wipes `C:\Program Files\GenieOpenAIService\`.
- Model files are not touched when they are outside the install directory (per config absolute paths).

## Supported APIs and Example Test Cases

In the current version of the Genie Service and Test App, we support the following API endpoints:

- POST /v1/chat/completions
- POST /v1/embeddings
- GET /v1/models

The following examples illustrate test cases for available APIs. Examples for all test cases are defined in `OpenAIClientApp_test_suite.json`; update the `model` / `model_id` fields to match the model IDs configured in the service.

**models\_list** — /v1/models — List all loaded models

> 
> 
> {
>       "id": "<test case id, e.g. list_models>",
>       "name": "<test case name, e.g. GET /v1/models returns at least one model>",
>       "type": "models_list",
>       "request": {},
>       "expected": {
>         "min_model_count": 1
>       }
>     }
>     Copy to clipboard

**models\_retrieve** — /v1/models/{model} — Retrieve a specific model by ID

> 
> 
> {
>       "id": "<test case id, e.g. retrieve_my_model>",
>       "name": "<test case name, e.g. GET /v1/models/meta-llama/llama3.2-3b>",
>       "type": "models_retrieve",
>       "request": {
>         "model": "<model ID to retrieve, e.g. meta-llama/llama3.2-3b>"
>       },
>       "expected": {
>         "object": "model",
>         "id_matches_request": true
>       }
>     }
>     Copy to clipboard

**chat\_completion** — /v1/chat/completions — Non-streaming chat completion

> 
> 
> {
>       "id": "<test case id, e.g. basic_chat>",
>       "name": "<test case name, e.g. Basic chat completion>",
>       "type": "chat_completion",
>       "request": {
>         "model": "<model ID to use, e.g. meta-llama/llama3.2-3b>",
>         "messages": [
>           {
>             "role": "<role to use, e.g. user>",
>             "content": "<message content, e.g. Say hello.>"
>           }
>         ],
>         "max_tokens": 20,
>         "temperature": 0.7
>       },
>       "expected": {
>         "object": "chat.completion",
>         "choices_count": 1,
>         "choice_role": "<expected role, e.g. assistant>",
>         "has_content": true,
>         "has_usage": true
>       }
>     }
>     Copy to clipboard

**chat\_completion\_stream** — /v1/chat/completions — Streaming chat completion

> 
> 
> {
>       "id": "<test case id, e.g. streaming_chat>",
>       "name": "<test case name, e.g. Streaming chat completion>",
>       "type": "chat_completion_stream",
>       "request": {
>         "model": "<model ID to use, e.g. meta-llama/llama3.2-3b>",
>         "messages": [
>           {
>             "role": "<role to use, e.g. user>",
>             "content": "<message content, e.g. Count to 5.>"
>           }
>         ],
>         "max_tokens": 30
>       },
>       "expected": {
>         "chunks_min": 1,
>         "full_text_not_empty": true
>       }
>     }
>     Copy to clipboard

**embedding** — /v1/embeddings — Generate text embeddings

> 
> 
> {
>       "id": "<test case id, e.g. single_embedding>",
>       "name": "<test case name, e.g. Single string embedding>",
>       "type": "embedding",
>       "request": {
>         "model": "<model ID to use, e.g. BAAI/bge-large-en>",
>         "input": "<input text to embed, e.g. Hello world>",
>         "encoding_format": "<encoding format to output, e.g. float>"
>       },
>       "expected": {
>         "embedding_count": 1,
>         "embedding_dim_min": 128,
>         "has_usage": true
>       }
>     }
>     Copy to clipboard

**parallel** — Concurrent sub-requests

> 
> 
> {
>       "id": "<test case id, e.g. parallel_chat_completions>",
>       "name": "<test case name, e.g. 4 concurrent chat completions>",
>       "type": "parallel",
>       "request": {
>         "sub_tests": [
>           {
>             "id": "<test case id, e.g. req_1>",
>             "type": "chat_completion",
>             "request": {
>               "model": "<model ID to use, e.g. meta-llama/llama3.2-3b>",
>               "messages": [{ "role": "user", "content": "Say: alpha" }],
>               "max_tokens": 10
>             },
>             "expected": { "has_content": true }
>           },
>           {
>             "id": "<test case id, e.g. req_2>",
>             "type": "chat_completion",
>             "request": {
>               "model": "<model ID to use, e.g. meta-llama/llama3.2-3b>",
>               "messages": [{ "role": "user", "content": "Say: beta" }],
>               "max_tokens": 10
>             },
>             "expected": { "has_content": true }
>           },
>           {
>             "id": "<test case id, e.g. req_3>",
>             "type": "chat_completion",
>             "request": {
>               "model": "<model ID to use, e.g. meta-llama/llama3.2-3b>",
>               "messages": [{ "role": "user", "content": "Say: gamma" }],
>               "max_tokens": 10
>             },
>             "expected": { "has_content": true }
>           },
>           {
>             "id": "<test case id, e.g. req_4>",
>             "type": "chat_completion",
>             "request": {
>               "model": "<model ID to use, e.g. meta-llama/llama3.2-3b>",
>               "messages": [{ "role": "user", "content": "Say: delta" }],
>               "max_tokens": 10
>             },
>             "expected": { "has_content": true }
>           }
>         ]
>       },
>       "expected": {
>         "all_pass": true,
>         "max_workers": 4
>       }
>     }
>     Copy to clipboard

## Genie Service Configuration

The Genie Service can be customized by modifying the service configuration file in the deployed package `config/` directory (for example, Windows uses `config/genie_openai_service_config.windows.json`; Linux provides an equivalent config file in the same location). The following is a breakdown of the sections in the config, the default configuration, and what modifications are possible:

- Note: The example config includes Linux file paths. For Windows deployment, adjust the paths appropriately.

### Version

- Version should not be modified manually besides during major Genie version updates

"version": "1.0"
        Copy to clipboard

### HTTP Server

- HTTP Server parameters can be configured based on need, ensure that only one service is running on a given port to avoid conflicts and requests timing out.

"server": {
          "host": "0.0.0.0",
          "port": 8080,
          "thread_pool_size": 16,
          "request_timeout_seconds": 300
        }
        Copy to clipboard

### Resource Limits

- Resource limits can be configured for this service to allow for flexible combinations of models loaded. Possible configuration options are listed below.
- - memory\_budget\_mb: Maximum memory allowed for the service process.
    - - Checked before each model load
    - If current usage exceeds threshold, evict LRU or reject load
    - Allows flexible model combinations (many small or few large)
    - Set to 0 to disable (no memory guard, load until NPU OOM)

"limits": {
          "memory_budget_mb": 12000,
          "eviction_policy": "lru"
        }
        Copy to clipboard

### Watchdog

- Watchdog can be configured to monitor for and cancel stuck Genie SDK calls.

"watchdog": {
          "enabled": true,
          "interval_seconds": 10,
          "stuck_threshold_seconds": 300
        }
        Copy to clipboard

### Models

- The models section contains the configuration for the individual models supported in this instance of the Genie service. Currently the service supports “llm”, “embedding”, and “lmm” model types:

"models": [
      {
        "model_id": "llm-qwen2.5-3b",
        "model_type": "llm",
        "provider": "genie",
        ...
      },
      {
        "model_id": "embedding-bge-large-en",
        "model_type": "embedding",
        "provider": "genie",
        ...
      },
      {
        "model_id": "internvl3.5-1b",
        "model_type": "lmm",
        "provider": "genie",
        ...
      }
    ]
    Copy to clipboard

### LLM

- model\_id can be chosen by the user but must be used consistently on both service and client side.
- model\_type must be llm and provider must be genie
- auto\_load defines whether the model is loaded during service initialization or during the first client invocation. true means load during service init, false means load during first client invocation
- queue-&gt;max\_depth defines how many requests for this model, from all clients, can be queued by the service. Any additional requests would be rejected by the service.
- provider\_config-&gt;config\_file must be a json file containing the Genie configuration for this model.
- provider\_config-&gt;genie\_log\_level is the model-specific logging level to be used when this model is invoked.
- defaults contains default inference parameters for cases where the client does not provide preferred parameters. temperature must be non-zero to avoid inference calls hanging.
- lora is optional, but if used lora-&gt;name must match the lora name defined in the Genie config file for this model. If a lora is defined it will be used for all inference calls, and cannot be turned on/off between calls.
- performance-&gt;htp\_perf\_profile allows the user to optionally configure the performance mode for inference calls. The performance mode chosen here will apply to all inference calls and cannot be customized between calls. Accepted values are burst, sustained\_high\_performance, high\_performance, balanced, low\_balanced, high\_power\_saver, low\_power\_saver, power\_saver, extreme\_power\_saver.
- prompting allows for configuring the default system prompt, system headers/footers, user headers/footers, and assistant headers/footers. These can be modified based on preferences and the model itself.

{
          "model_id": "llm-qwen2.5-3b",
          "model_type": "llm",
          "provider": "genie",
          "auto_load": true,
          "memory_estimate_mb": 4096,
          "queue": {
            "max_depth": 16
          },
          "provider_config": {
            "config_file": "/tmp/qaior-runtime/models/llm-qwen2.5-3b/genie_config.json",
            "genie_log_level": "verbose"
          },
          "defaults": {
            "temperature": 1.0,
            "top_p": 1.0,
            "max_tokens": 2048,
            "stop_sequences": [
              "<|eot_id|>",
              "<|eom_id|>"
            ]
          },
          "performance": {
            "htp_perf_profile": "balanced"
          },
          "prompting": {
            "default_system_prompt": "You are a helpful, respectful, and honest assistant.\nAlways answer as helpfully as possible.\n",
            "system_header": "<|start_header_id|>system<|end_header_id|>\n\n",
            "system_footer": "<|eot_id|>",
            "user_header": "<|start_header_id|>user<|end_header_id|>\n\n",
            "user_footer": "<|eot_id|>",
            "assistant_header": "<|start_header_id|>assistant<|end_header_id|>\n\n",
            "assistant_footer": "<|eot_id|>"
          },
          "session_strategy": "single"
        }
        Copy to clipboard

### Embedding

- model\_id can be chosen by the user but must be used consistently on both service and client side.
- model\_type must be embedding and provider must be genie
- auto\_load defines whether the model is loaded during service initialization or during the first client invocation. true means load during service init, false means load during first client invocation
- queue-&gt;max\_depth defines how many requests for this model, from all clients, can be queued by the service. Any additional requests would be rejected by the service.
- provider\_config-&gt;config\_file must be a json file containing the Genie configuration for this model.
- provider\_config-&gt;genie\_log\_level is the model-specific logging level to be used when this model is invoked.
- performance-&gt;htp\_perf\_profile allows the user to optionally configure the performance mode for inference calls. The performance mode chosen here will apply to all inference calls and cannot be customized between calls. Accepted values are burst, sustained\_high\_performance, high\_performance, balanced, low\_balanced, high\_power\_saver, low\_power\_saver, power\_saver, extreme\_power\_saver.

{
          "model_id": "embedding-bge-large-en",
          "model_type": "embedding",
          "provider": "genie",
          "auto_load": true,
          "memory_estimate_mb": 4096,
          "queue": {
            "max_depth": 32
          },
          "provider_config": {
            "config_file": "/tmp/qaior-runtime/models/embedding-bge-large-en/genie_config.json",
            "genie_log_level": "verbose"
          },
          "performance": {
            "htp_perf_profile": "balanced"
          }
        }
        Copy to clipboard

### LMM

- model\_id can be chosen by the user but must be used consistently on both service and client side.
- model\_type must be lmm and provider must be genie
- auto\_load defines whether the model is loaded during service initialization or during the first client invocation. true means load during service init, false means load during first client invocation
- queue-&gt;max\_depth defines how many requests for this model, from all clients, can be queued by the service. Any additional requests would be rejected by the service.
- provider\_config-&gt;config\_file must be a json file containing the Genie configuration for this model.
- provider\_config-&gt;genie\_log\_level is the model-specific logging level to be used when this model is invoked.
- performance-&gt;htp\_perf\_profile allows the user to optionally configure the performance mode for inference calls. The performance mode chosen here will apply to all inference calls and cannot be customized between calls. Accepted values are burst, sustained\_high\_performance, high\_performance, balanced, low\_balanced, high\_power\_saver, low\_power\_saver, power\_saver, extreme\_power\_saver.
- image\_preprocessing\_config is an optional configuration file to configure image pre-processing for this LMM model.
- prompting allows for configuring the default system prompt, system headers/footers, user headers/footers, assistant headers/footers, and image headers/footers. These can be modified based on preferences and the model itself.

{
          "model_id": "internvl3.5-1b",
          "model_type": "lmm",
          "provider": "genie",
          "auto_load": false,
          "session_strategy": "single",
          "queue": {
            "max_depth": 4
          },
          "provider_config": {
            "config_file": "/tmp/qaior-runtime/models/lmm-internvl3_5-1b/pipeline_config.json",
            "genie_log_level": "info"
          },
          "performance": {
            "htp_perf_profile": "balanced"
          },
          "image_preprocessing_config": "/tmp/qaior-runtime/models/lmm-internvl3_5-1b/image_preprocessing_config.yaml",
          "prompting": {
            "default_system_prompt": "You are a helpful visual assistant.",
            "system_header": "<|im_start|>system\n",
            "system_footer": "<|im_end|>\n",
            "user_header": "<|im_start|>user\n",
            "user_footer": "<|im_end|>\n",
            "assistant_header": "<|im_start|>assistant\n",
            "assistant_footer": "<|im_end|>",
            "image_header": "",
            "image_footer": ""
          }
        }
        Copy to clipboard

## Genie Model Configuration

Any models used by the Genie Service must include a Genie model configuration file alongside the model binaries. Here is an example Genie model configuration for reference, ensure that the tokenizer path, extensions path, and ctx-bins point to the correct model files:

Example Genie model configs can be found here: `${QAIRT_SDK_ROOT}/examples/Genie/configs`

Last Published: Sep 03, 2026

[Previous Topic
Genie DLC](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/dlc.md) [Next Topic
Library](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/library.md)