---
title: "Use Augure with LiteLLM"
description: "Call Augure's Canadian, OpenAI-compatible API through the LiteLLM SDK or proxy with the openai/ prefix and a custom api_base."
tool: "LiteLLM"
category: "framework"
canonical: "https://augureai.ca/docs/integrations/litellm"
updated: "2026-10-26"
verified: false
---
# Use Augure with LiteLLM

You get Augure as a provider in LiteLLM, both in the Python SDK and in the LiteLLM proxy, with inference on Canadian infrastructure.

## Prerequisites

- Python 3.8+ and `pip install litellm` (add `pip install 'litellm[proxy]'` for the proxy).
- An Augure API key (`aug_sk_live_…`) from the [developer dashboard](https://chat.augureai.ca/developers/dashboard). Plans and pricing: [augureai.ca/developers](https://augureai.ca/developers).
- The key exported as `AUGURE_API_KEY`.

## Configure

### Python SDK

Prefix the model id with `openai/` so LiteLLM uses the OpenAI Chat Completions format, and set `api_base`.

```python test
import os
from litellm import completion

response = completion(
    model="openai/tofino-3",
    api_base="https://api.augureai.ca/v1",
    api_key=os.environ["AUGURE_API_KEY"],
    messages=[{"role": "user", "content": "Say hello in one short sentence."}],
    max_tokens=64,
)
print(response.choices[0].message.content)
```

Streaming works with `stream=True` and `stream_options={"include_usage": True}`; iterate the chunks as with the OpenAI SDK.

### Proxy

```yaml
# config.yaml
model_list:
  - model_name: augure-ossington-5
    litellm_params:
      model: openai/ossington-5
      api_base: https://api.augureai.ca/v1
      api_key: os.environ/AUGURE_API_KEY
  - model_name: augure-rosedale-1
    litellm_params:
      model: openai/rosedale-1
      api_base: https://api.augureai.ca/v1
      api_key: os.environ/AUGURE_API_KEY
  - model_name: augure-tofino-3
    litellm_params:
      model: openai/tofino-3
      api_base: https://api.augureai.ca/v1
      api_key: os.environ/AUGURE_API_KEY
```

```bash
litellm --config config.yaml
```

Clients then call the proxy (`http://localhost:4000/v1` by default) with `model: "augure-tofino-3"` and the proxy's own key.

## Pick a model

| Model | When to use |
| --- | --- |
| `ossington-5` | Flagship reasoning. 256K context, 32,768 max output. Default for hard tasks. |
| `rosedale-1` | Premium reasoning, up to 1M context on paid plans, Canadian inference. Long documents, agent loops. |
| `tofino-3` | Fast everyday model, 256K context, 16,384 max output, no thinking. Chat, extraction, classification. |

`openai/auto` lets the gateway pick. On `ossington-5`, `reasoning_effort="none"` turns thinking off and is the fastest setting for mechanical tasks; `low`, `medium` and `high` are accepted on reasoning models; `tofino-3` ignores it. LiteLLM forwards `reasoning_effort` to OpenAI-compatible providers as a keyword argument. Full list: [/docs/models](https://augureai.ca/docs/models).

## Verify it works

Check residency with one direct call:

```bash test
curl -s https://api.augureai.ca/v1/chat/completions \
  -H "Authorization: Bearer $AUGURE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tofino-3","max_tokens":64,"messages":[{"role":"user","content":"Reply with the single word OK."}]}'
```

The JSON ends with an `_augure` block, for example `"_augure": {"gateway_region": "ca-montreal-1", "inference_region": "ca-on-toronto", "request_id": "..."}`. `inference_region` tells you where the model ran. Inference is on Canadian infrastructure first, with disclosed failover to vetted EU partners under zero data retention, never the US. LiteLLM re-serializes responses into its own `ModelResponse`, so the `_augure` block may not survive the round trip through the SDK or proxy; use the direct call above when you need it.

## Notes

- Without the `openai/` prefix LiteLLM tries to guess the provider from the model name and fails on Augure ids.
- Tool calling (`tools=[...]`) and `response_format` pass straight through.
- Reasoning text from thinking models comes back in `reasoning_content`; LiteLLM surfaces it as `message.reasoning_content` in recent versions.
- OpenAI model aliases are accepted at the gateway (`gpt-4o` maps to `ossington-4`, `gpt-4o-mini` to `tofino-3`), but name the Augure model explicitly in `config.yaml` so routing is obvious.

## See also

- [API reference](https://augureai.ca/docs/api)
- [Models](https://augureai.ca/docs/models)
- [MCP server](https://augureai.ca/docs/mcp)
- [IDE setup](https://augureai.ca/docs/ide)
- [Pricing](https://augureai.ca/pricing)