Canadian inference.
Competitive performance on Canadian work.
We evaluated 7 models across 4 Canadian work suites covering grounded answers, French instruction-following and structured extraction. Ossington 5 scored 97.3 on the published-suite average, compared with 97.8 for our Claude Sonnet 5.5 reference. Both Augure models also matched the reference’s 60/60 score on our bilingual Canadian document-answering evaluation.
Evaluations authored and run by Augure. The bilingual suite uses supplied documents and fictional scenarios based on Canadian rules; 6 of 7 models scored 60/60.
Rosedale One · General-purpose intelligence. Canadian inference.
Strong measured performance across coding, mathematics, scientific reasoning and knowledge. In our October evaluations, Rosedale One scored 98.2 on HumanEval (joint-highest in our comparison) and 86.7 on AIME 2025, with all published evaluation samples served from Canada.
98.2
Coding · HumanEval
joint-highest (2 models) · n = 164 · CI 94.8–99.4
86.7
Math · AIME 2025
3rd of 7 · n = 30 · CI 70.3–94.7
83.0
Reasoning · GPQA Diamond
5th of 7 · n = 792 · CI 78.3–87.6
86.0
Knowledge · MMLU-Pro
5th of 7 · n = 500 · CI 82.7–88.8
$6.00 per million output tokens, 58 % below the listed Claude Sonnet 5.5 rate; input $2.50, 12 % below. Actual savings depend on the workload. · dated source
Ossington 5 · Everyday Canadian work at a lower price.
Ossington 5 scored 97.3 on average across the 4 published work suites, against 97.8 for the Claude Sonnet 5.5 reference, including 60/60 on grounded answers from Canadian documents.
98.3
Grounded answers · Québec sources
2nd of 7 · n = 60 · CI 91.1–99.7
100.0
Grounded answers · Canadian law & public services
joint-highest (6 models) · n = 60 · CI 94.0–100.0
95.0
French instructions · IFEval-QC
3rd of 7 · n = 60 · CI 86.3–98.3
96.0
Structured extraction · Québec documents
5th of 7 · n = 50 · CI 86.5–98.9
$3.00 per million output tokens, 79 % below the listed Claude Sonnet 5.5 rate; input $1.50, 47 % below. Actual savings depend on the workload. · dated source
Inference location: every Augure call in the published runs carries its serving region in the response (Waterloo or Toronto, Ontario); the evaluation key is configured to fail rather than leave Canada. Other models are called through their vendors' public APIs.
October 2026 · 7 models · 4 work suites · 230 tasksDownload aggregates (JSON)
Four suites authored and run by Augure, 230 tasks: answering from supplied documents (in French on Québec sources; in English and French on documents from across Canada), following French instructions, and extracting structured data from Québec documents. Gaps of a few points between overlapping intervals do not establish superiority.
AugureOther models95% confidence intervalShared 70–100 axis
Grounded answers · Québec sources
Rosedale One scored 60/60; Ossington 5 matched the Claude Sonnet 5.5 reference at 59/60.
60 tasks · score / 100 · 95% CI · axis 70–100
View data table
| Model | Score | Correct | n | 95% CI |
|---|---|---|---|---|
| Rosedale One | 100.0 | 60 | 60 | 94.0 – 100.0 |
| Claude Haiku 5.5 | 98.3 | 59 | 60 | 91.1 – 99.7 |
| Claude Sonnet 5.5 | 98.3 | 59 | 60 | 91.1 – 99.7 |
| Gemini 3.8 Flash | 98.3 | 59 | 60 | 91.1 – 99.7 |
| Mistral Large 4 | 98.3 | 59 | 60 | 91.1 – 99.7 |
| Ossington 5 | 98.3 | 59 | 60 | 91.1 – 99.7 |
| GPT-5.4 mini | 90.0 | 54 | 60 | 79.9 – 95.3 |
Grounded answers · Canadian law & public services
Both Augure models scored 60/60, alongside 4 other models.
60 tasks · score / 100 · 95% CI · axis 70–100
View data table
| Model | Score | Correct | n | 95% CI |
|---|---|---|---|---|
| Claude Haiku 5.5 | 100.0 | 60 | 60 | 94.0 – 100.0 |
| Claude Sonnet 5.5 | 100.0 | 60 | 60 | 94.0 – 100.0 |
| Gemini 3.8 Flash | 100.0 | 60 | 60 | 94.0 – 100.0 |
| Mistral Large 4 | 100.0 | 60 | 60 | 94.0 – 100.0 |
| Ossington 5 | 100.0 | 60 | 60 | 94.0 – 100.0 |
| Rosedale One | 100.0 | 60 | 60 | 94.0 – 100.0 |
| GPT-5.4 mini | 98.3 | 59 | 60 | 91.1 – 99.7 |
French instructions · IFEval-QC
Ossington 5 matched the Claude Sonnet 5.5 reference at 57/60; Rosedale One scored 54/60.
60 tasks · score / 100 · 95% CI · axis 70–100
View data table
| Model | Score | Correct | n | 95% CI |
|---|---|---|---|---|
| Mistral Large 4 | 100.0 | 60 | 60 | 94.0 – 100.0 |
| Gemini 3.8 Flash | 98.3 | 59 | 60 | 91.1 – 99.7 |
| Claude Sonnet 5.5 | 95.0 | 57 | 60 | 86.3 – 98.3 |
| GPT-5.4 mini | 95.0 | 57 | 60 | 86.3 – 98.3 |
| Ossington 5 | 95.0 | 57 | 60 | 86.3 – 98.3 |
| Claude Haiku 5.5 | 90.0 | 54 | 60 | 79.9 – 95.3 |
| Rosedale One | 90.0 | 54 | 60 | 79.9 – 95.3 |
Structured extraction · Québec documents
Ossington 5 scored 48/50; Rosedale One scored 46/50, compared with 49/50 for the Claude Sonnet 5.5 reference.
50 tasks · score / 100 · 95% CI · axis 70–100
View data table
| Model | Score | Correct | n | 95% CI |
|---|---|---|---|---|
| Claude Haiku 5.5 | 98.0 | 49 | 50 | 89.5 – 99.6 |
| Claude Sonnet 5.5 | 98.0 | 49 | 50 | 89.5 – 99.6 |
| Gemini 3.8 Flash | 98.0 | 49 | 50 | 89.5 – 99.6 |
| GPT-5.4 mini | 98.0 | 49 | 50 | 89.5 – 99.6 |
| Ossington 5 | 96.0 | 48 | 50 | 86.5 – 98.9 |
| Mistral Large 4 | 94.0 | 47 | 50 | 83.8 – 97.9 |
| Rosedale One | 92.0 | 46 | 50 | 81.2 – 96.8 |
Work-suite score versus listed output-token price
Higher scores and lower listed output prices appear toward the upper left.
Vertical axis is a focused 90–100 range: small differences do not establish superiority. Equal-weight mean of the published work suites. Prices in CAD per million tokens.
| Model | Score | Input (CAD / M) | Output (CAD / M) |
|---|---|---|---|
| Gemini 3.8 Flash | 98.7 | $1.07 | $5.35 |
| Mistral Large 4 | 98.1 | $1.94 | $5.96 |
| Claude Sonnet 5.5Reference | 97.8 | $2.85 | $14.26 |
| Ossington 5 | 97.3 | $1.50 | $3.00 |
| Claude Haiku 5.5 | 96.6 | $0.14 | $0.71 |
| Rosedale One | 95.5 | $2.50 | $6.00 |
| GPT-5.4 mini | 95.3 | $1.07 | $6.42 |
Vendors' listed prices (standard tier, no batch or cache rates) as of 2026-10-14, converted to CAD at the Bank of Canada rate (1.4263). Augure: developer API list price. Token price is not task cost: billed reasoning, retries and failures are not counted here.