Use Augure with LiteLLM

You get Augure as a provider in LiteLLM, both in the Python SDK and in the LiteLLM proxy, with inference on Canadian infrastructure.

Prerequisites

• Python 3.8+ and pip install litellm (add pip install 'litellm[proxy]' for the proxy).
• An Augure API key (aug_sk_live_…) from the developer dashboard. Plans and pricing: augureai.ca/developers.
• The key exported as AUGURE_API_KEY.

Configure

Python SDK

Prefix the model id with openai/ so LiteLLM uses the OpenAI Chat Completions format, and set api_base.

import os
from litellm import completion

response = completion(
    model="openai/tofino-3",
    api_base="https://api.augureai.ca/v1",
    api_key=os.environ["AUGURE_API_KEY"],
    messages=[{"role": "user", "content": "Say hello in one short sentence."}],
    max_tokens=64,
)
print(response.choices[0].message.content)

Streaming works with stream=True and stream_options={"include_usage": True}; iterate the chunks as with the OpenAI SDK.

Proxy

# config.yaml
model_list:
  - model_name: augure-ossington-5
    litellm_params:
      model: openai/ossington-5
      api_base: https://api.augureai.ca/v1
      api_key: os.environ/AUGURE_API_KEY
  - model_name: augure-rosedale-1
    litellm_params:
      model: openai/rosedale-1
      api_base: https://api.augureai.ca/v1
      api_key: os.environ/AUGURE_API_KEY
  - model_name: augure-tofino-3
    litellm_params:
      model: openai/tofino-3
      api_base: https://api.augureai.ca/v1
      api_key: os.environ/AUGURE_API_KEY

litellm --config config.yaml

Clients then call the proxy (http://localhost:4000/v1 by default) with model: "augure-tofino-3" and the proxy's own key.

Pick a model

Model	When to use
ossington-5	Flagship reasoning. 256K context, 32,768 max output. Default for hard tasks.
rosedale-1	Premium reasoning, up to 1M context on paid plans, Canadian inference. Long documents, agent loops.
tofino-3	Fast everyday model, 256K context, 16,384 max output, no thinking. Chat, extraction, classification.

openai/auto lets the gateway pick. On ossington-5, reasoning_effort="none" turns thinking off and is the fastest setting for mechanical tasks; low, medium and high are accepted on reasoning models; tofino-3 ignores it. LiteLLM forwards reasoning_effort to OpenAI-compatible providers as a keyword argument. Full list: /docs/models.

Verify it works

Check residency with one direct call:

curl -s https://api.augureai.ca/v1/chat/completions \
  -H "Authorization: Bearer $AUGURE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tofino-3","max_tokens":64,"messages":[{"role":"user","content":"Reply with the single word OK."}]}'

The JSON ends with an _augure block, for example "_augure": {"gateway_region": "ca-montreal-1", "inference_region": "ca-on-toronto", "request_id": "..."}. inference_region tells you where the model ran. Inference is on Canadian infrastructure first, with disclosed failover to vetted EU partners under zero data retention, never the US. LiteLLM re-serializes responses into its own ModelResponse, so the _augure block may not survive the round trip through the SDK or proxy; use the direct call above when you need it.

Notes

• Without the openai/ prefix LiteLLM tries to guess the provider from the model name and fails on Augure ids.
• Tool calling (tools=[...]) and response_format pass straight through.
• Reasoning text from thinking models comes back in reasoning_content; LiteLLM surfaces it as message.reasoning_content in recent versions.
• OpenAI model aliases are accepted at the gateway (gpt-4o maps to ossington-4, gpt-4o-mini to tofino-3), but name the Augure model explicitly in config.yaml so routing is obvious.

See also

• API reference
• Models
• MCP server
• IDE setup
• Pricing