Use Augure with LlamaIndex

You get Augure as the LLM behind LlamaIndex query engines, chat engines and agents, with inference on Canadian infrastructure.

Prerequisites

• Python 3.9+ and pip install llama-index llama-index-llms-openai-like.
• An Augure API key (aug_sk_live_…) from the developer dashboard. Plans and pricing: augureai.ca/developers.
• The key exported as AUGURE_API_KEY.

Configure

Use OpenAILike, not the plain OpenAI LLM class. The OpenAI class checks model names against OpenAI's own list and rejects tofino-3; OpenAILike sends the same requests to any OpenAI-compatible base URL and lets you declare the model's capabilities.

import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="tofino-3",
    api_base="https://api.augureai.ca/v1",
    api_key=os.environ["AUGURE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=256000,
    max_tokens=64,
)

print(llm.complete("Say hello in one short sentence.").text)

Make it the default for every index and engine:

from llama_index.core import Settings
Settings.llm = llm

Streaming:

for delta in llm.stream_complete("Hello"):
    print(delta.delta, end="", flush=True)

is_function_calling_model=True enables FunctionAgent and other tool-calling agents; is_chat_model=True makes LlamaIndex use /v1/chat/completions. Set context_window to the model you chose (256000 for ossington-5 and tofino-3; rosedale-1 goes up to 1,000,000 on paid plans).

Pick a model

Model	When to use
ossington-5	Flagship reasoning. 256K context, 32,768 max output. Default for hard tasks.
rosedale-1	Premium reasoning, up to 1M context on paid plans, Canadian inference. Long documents, agent loops.
tofino-3	Fast everyday model, 256K context, 16,384 max output, no thinking. Chat, extraction, classification.

model="auto" lets the gateway pick. On ossington-5, reasoning_effort="none" turns thinking off and is the fastest setting for mechanical tasks; low, medium and high are accepted on reasoning models; tofino-3 ignores it. Pass it through additional_kwargs={"reasoning_effort": "none"} on OpenAILike. Full list: /docs/models.

Verify it works

Check residency with one direct call:

curl -s https://api.augureai.ca/v1/chat/completions \
  -H "Authorization: Bearer $AUGURE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tofino-3","max_tokens":64,"messages":[{"role":"user","content":"Reply with the single word OK."}]}'

The JSON ends with an _augure block, for example "_augure": {"gateway_region": "ca-montreal-1", "inference_region": "ca-on-toronto", "request_id": "..."}. inference_region tells you where the model ran. Inference is on Canadian infrastructure first, with disclosed failover to vetted EU partners under zero data retention, never the US. In LlamaIndex, the raw attribute of a response holds the full API response object when the LLM class keeps it.

Notes

• Embeddings are a separate endpoint and are not covered here; keep your existing embedding model for indexes.
• response.text and chat_response.message.content hold the final answer only; reasoning text from thinking models stays in the API's reasoning_content field.
• OpenAI model aliases are accepted: gpt-4o maps to ossington-4, gpt-4o-mini to tofino-3.
• OpenAILike also accepts timeout and max_retries; the defaults are fine for most workloads.

See also

• API reference
• Models
• MCP server
• IDE setup
• Pricing