# Use LiteLLM to abstract API calls

[LiteLLM](https://docs.litellm.ai/docs/) is a popular framework to call LLM APIs using the OpenAI format to abstract API calls. Use `litellm` to call the Imagine Playground APIs.

## Prerequisites

- Complete the [get started prerequisites](https://docs.qualcomm.com/doc/80-88545-1/topic/litellm.html#prerequistes).
- LiteLLM installed. If not already installed, do the following:

pip install litellm
        Copy to clipboard

## Start a chat with the LLM (non-streaming)

The following example shows how to start a chat with a large language model (LLM) using LiteLLM. The `litellm.completion` function specifies the LLM to use and the messages to pass to the LLM. The response is printed after the entire response is received.

from rich.pretty import pprint
    
    import litellm

    # litellm.api_base=""
    # litellm.api_key=""
    
    response = litellm.completion(
        model="openai/Llama-3.1-8B",
        messages=[
            {
                "role": "user",
                "content": "Tell me about the new wonders of the world",
            }
        ],
        max_tokens=1024,
    )
    
    pprint(response.choices[0].message.content)
    Copy to clipboard

The output is similar to the following:

You're referring to the New 7 Wonders of the World, a list compiled in 2007 through a worldwide poll. Here are the seven wonders:
    
    1. **The Great Wall of China**: A series of fortifications built across several Chinese dynasties to protect the country from invasions. It stretches over 13,000 miles (21,000 km) and is one of the longest structures ever built.
    2. **The Taj Mahal** (India): A mausoleum built by Mughal Emperor Shah Jahan in memory of his wife, Mumtaz Mahal. It's a stunning example of Mughal architecture, with intricate marble inlays and a perfect blend of Indian, Persian, and Islamic styles.
    3. **Machu Picchu** (Peru): An Inca citadel built in the 15th century, abandoned before the arrival of the Spanish conquistadors, and rediscovered in the 20th century. It's a testament to the engineering and architectural skills of the Incas.
    4. **Chichén Itzá** (Mexico): A pre-Columbian Mayan city on the Yucatán Peninsula, built by the Mayans around 1100 AD. It features the Pyramid of Kukulkan, a 98-foot-tall (30 meters) pyramid that aligns with the movements of the sun and the stars.
    5. **The Roman Colosseum** (Italy): An amphitheater built in the 1st century AD, hosting gladiatorial contests, animal hunts, and public spectacles. It's a symbol of the power and engineering prowess of the ancient Romans.
    6. **The Christ the Redeemer statue** (Brazil): A massive Art Deco statue of Jesus Christ, built in Rio de Janeiro between 1922 and 1931. It's 98 feet (30 meters) tall, with outstretched arms and a serene expression.
    7. **The Pyramids of Giza** (Egypt): The only remaining ancient wonder from the original list, these pyramids are the oldest and only remaining structure from the original list of Seven Wonders of the Ancient World. The largest pyramid, the Great Pyramid of Giza, is an astonishing 481 feet (147 meters) tall.
    
    ...
    Copy to clipboard

## Start a chat with the LLM (streaming)

This example is similar to the previous example except that it prints each part of the LLM’s response in real-time so that you can start providing feedback to the user as soon as possible rather than waiting for the entire response. As with the previous example, this example also uses the`litellm.completion` function to specify the LLM to use and the messages to pass to the LLM. This example adds `stream=True` to the function so that the text displays as each part of the response is received. This approach is useful to reduce perceived latency.

import litellm

    # litellm.api_base=""
    # litellm.api_key=""
    
    stream_response = litellm.completion(
        model="openai/Llama-3.1-8B",
        messages=[
            {
                "role": "user",
                "content": "Tell me about the new wonders of the world",
            }
        ],
        max_tokens=500,
        stream=True,
    )
    
    for chunk in stream_response:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    Copy to clipboard

The output is similar to the following:

You're referring to the New 7 Wonders of the World, a list compiled in 2007 through a worldwide poll. Here are the seven wonders:
    
    1. **The Great Wall of China**: A series of fortifications built across several Chinese dynasties to protect the country from invasions. It stretches over 13,000 miles (21,000 km) and is one of the longest structures ever built.
    2. **The Taj Mahal** (India): A mausoleum built by Mughal Emperor Shah Jahan in memory of his wife, Mumtaz Mahal. It's a stunning example of Mughal architecture, with intricate marble inlays and a perfect blend of Indian, Persian, and Islamic styles.
    3. **Machu Picchu** (Peru): An Inca citadel built in the 15th century, abandoned before the arrival of the Spanish conquistadors, and rediscovered in the 20th century. It's a testament to the engineering and architectural skills of the Incas.
    4. **Chichén Itzá** (Mexico): A pre-Columbian Mayan city on the Yucatán Peninsula, built by the Mayans around 1100 AD. It features the Pyramid of Kukulkan, a 98-foot-tall (30 meters) pyramid that aligns with the movements of the sun and the stars.
    5. **The Roman Colosseum** (Italy): An amphitheater built in the 1st century AD, hosting gladiatorial contests, animal hunts, and public spectacles. It's a symbol of the power and engineering prowess of the ancient Romans.
    6. **The Christ the Redeemer statue** (Brazil): A massive Art Deco statue of Jesus Christ, built in Rio de Janeiro between 1922 and 1931. It's 98 feet (30 meters) tall, with outstretched arms and a serene expression.
    7. **The Pyramids of Giza** (Egypt): The only remaining ancient wonder from the original list, these pyramids are the oldest and only remaining structure from the original list of Seven Wonders of the Ancient World. The largest pyramid, the Great Pyramid of Giza, is an astonishing 481 feet (147 meters) tall.
    
    ...
    Copy to clipboard

## Code completion

This example invokes the `text_completion` method to generate code in response to a prompt.

from rich.pretty import pprint
    
    import litellm

    # litellm.api_base=""
    # litellm.api_key=""
    
    completion_response = litellm.text_completion(
        prompt="def fibonacci(n):",
        model="text-completion-openai/Llama-3.1-8B",
        max_tokens=100,
    )
    
    pprint(completion_response.choices[0].text)
    Copy to clipboard

The output is similar to the following:

\n    if n <= 0:\n        return "Input should be a positive integer"\n    elif n == 1:\n        return 0\n    elif n == 2:\n        return 1\n    else:\n        a, b = 0, 1\n        for _ in range(2, n):\n            a, b = b, a + b\n        return b\n ...
    Copy to clipboard

## Generate embeddings

Embeddings are numerical representations of text that capture semantic meaning. This code shows how to use LiteLLM to generate numerical representations (embeddings) for the text strings provided in the `litellm.embedding()`.

from rich.pretty import pprint
    
    import litellm

    litellm.suppress_debug_info = True
    
    # litellm.api_base=""
    # litellm.api_key=""
    
    response = litellm.embedding(
        model="openai/BAAI/bge-large-en-v1.5", input=["This is amazing", "Quite good stuff"]
    )
    
    pprint(response)
    Copy to clipboard

## Next steps

- Learn how to [configure logging to debug applications](https://docs.qualcomm.com/doc/80-88545-1/topic/6_0_logging.html).
- Review [basic functionality on how to start a chat, complete code, and generate embeddings](https://docs.qualcomm.com/doc/80-88545-1/topic/1_0_basic_usage.html).

Last Published: Apr 17, 2026

[Previous Topic
Use LangChain with the Qualcomm AI Inference Suite SDK](https://docs.qualcomm.com/bundle/publicresource/80-88545-1/topics/3_0_langchain.md) [Next Topic
Use CrewAI to create agent teams](https://docs.qualcomm.com/bundle/publicresource/80-88545-1/topics/crewai.md)