---
title: "Use Augure with LlamaIndex"
description: "Run LlamaIndex query engines and agents on Augure's Canadian, OpenAI-compatible API with the OpenAILike LLM class."
tool: "LlamaIndex"
category: "framework"
canonical: "https://augureai.ca/docs/integrations/llamaindex"
updated: "2026-10-26"
verified: false
---
# Use Augure with LlamaIndex

You get Augure as the LLM behind LlamaIndex query engines, chat engines and agents, with inference on Canadian infrastructure.

## Prerequisites

- Python 3.9+ and `pip install llama-index llama-index-llms-openai-like`.
- An Augure API key (`aug_sk_live_…`) from the [developer dashboard](https://chat.augureai.ca/developers/dashboard). Plans and pricing: [augureai.ca/developers](https://augureai.ca/developers).
- The key exported as `AUGURE_API_KEY`.

## Configure

Use `OpenAILike`, not the plain `OpenAI` LLM class. The `OpenAI` class checks model names against OpenAI's own list and rejects `tofino-3`; `OpenAILike` sends the same requests to any OpenAI-compatible base URL and lets you declare the model's capabilities.

```python test
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="tofino-3",
    api_base="https://api.augureai.ca/v1",
    api_key=os.environ["AUGURE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=256000,
    max_tokens=64,
)

print(llm.complete("Say hello in one short sentence.").text)
```

Make it the default for every index and engine:

```python
from llama_index.core import Settings
Settings.llm = llm
```

Streaming:

```python
for delta in llm.stream_complete("Hello"):
    print(delta.delta, end="", flush=True)
```

`is_function_calling_model=True` enables `FunctionAgent` and other tool-calling agents; `is_chat_model=True` makes LlamaIndex use `/v1/chat/completions`. Set `context_window` to the model you chose (256000 for `ossington-5` and `tofino-3`; `rosedale-1` goes up to 1,000,000 on paid plans).

## Pick a model

| Model | When to use |
| --- | --- |
| `ossington-5` | Flagship reasoning. 256K context, 32,768 max output. Default for hard tasks. |
| `rosedale-1` | Premium reasoning, up to 1M context on paid plans, Canadian inference. Long documents, agent loops. |
| `tofino-3` | Fast everyday model, 256K context, 16,384 max output, no thinking. Chat, extraction, classification. |

`model="auto"` lets the gateway pick. On `ossington-5`, `reasoning_effort="none"` turns thinking off and is the fastest setting for mechanical tasks; `low`, `medium` and `high` are accepted on reasoning models; `tofino-3` ignores it. Pass it through `additional_kwargs={"reasoning_effort": "none"}` on `OpenAILike`. Full list: [/docs/models](https://augureai.ca/docs/models).

## Verify it works

Check residency with one direct call:

```bash test
curl -s https://api.augureai.ca/v1/chat/completions \
  -H "Authorization: Bearer $AUGURE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tofino-3","max_tokens":64,"messages":[{"role":"user","content":"Reply with the single word OK."}]}'
```

The JSON ends with an `_augure` block, for example `"_augure": {"gateway_region": "ca-montreal-1", "inference_region": "ca-on-toronto", "request_id": "..."}`. `inference_region` tells you where the model ran. Inference is on Canadian infrastructure first, with disclosed failover to vetted EU partners under zero data retention, never the US. In LlamaIndex, the `raw` attribute of a response holds the full API response object when the LLM class keeps it.

## Notes

- Embeddings are a separate endpoint and are not covered here; keep your existing embedding model for indexes.
- `response.text` and `chat_response.message.content` hold the final answer only; reasoning text from thinking models stays in the API's `reasoning_content` field.
- OpenAI model aliases are accepted: `gpt-4o` maps to `ossington-4`, `gpt-4o-mini` to `tofino-3`.
- `OpenAILike` also accepts `timeout` and `max_retries`; the defaults are fine for most workloads.

## See also

- [API reference](https://augureai.ca/docs/api)
- [Models](https://augureai.ca/docs/models)
- [MCP server](https://augureai.ca/docs/mcp)
- [IDE setup](https://augureai.ca/docs/ide)
- [Pricing](https://augureai.ca/pricing)