Benchmarks
How leading models score on standardized task suites, with the average cost per task at LokaRouter prices.
τ²-Bench Airline Agents
12 models · 11,582 tasks · last run 2026-09-03
Multi-turn service agents making tool calls under strict policy constraints.
| Rank | Model | Accuracy | Avg cost / task |
|---|---|---|---|
| 1 | OpenAI: GPT-5.2 | 82.1% | $0.34 |
| 2 | Anthropic: Claude Opus 5 | 79.4% | $0.51 |
| 3 | Anthropic: Claude Sonnet 4.6 | 75.2% | $0.18 |
| 4 | Google: Gemini 3 Pro | 73.9% | $0.22 |
| 5 | OpenAI: GPT-5.1 | 70.8% | $0.16 |
| 6 | xAI: Grok 4.6 | 68.1% | $0.19 |
| 7 | Qwen: Qwen3 Max | 64.4% | $0.09 |
| 8 | DeepSeek: V4 | 61.7% | $0.05 |
GPQA Diamond Reasoning
15 models · 24,300 tasks · last run 2026-09-05
Graduate-level science questions that resist retrieval and reward careful reasoning.
| Rank | Model | Accuracy | Avg cost / task |
|---|---|---|---|
| 1 | OpenAI: GPT-5.2 Pro | 91.7% | $1.42 |
| 2 | OpenAI: GPT-5.2 | 89.3% | $0.38 |
| 3 | Anthropic: Claude Opus 5 | 88.1% | $0.55 |
| 4 | Google: Gemini 3 Pro | 87.4% | $0.24 |
| 5 | MoonshotAI: Kimi K3 | 86.2% | $0.11 |
| 6 | xAI: Grok 4.2 | 85.5% | $0.21 |
| 7 | DeepSeek: R2 | 84.8% | $0.07 |
| 8 | Anthropic: Claude Sonnet 4.6 | 83.6% | $0.19 |
| 9 | Qwen: Qwen3.8 Max | 82.9% | $0.10 |
| 10 | OpenAI: o3 | 81.7% | $0.29 |
BrowseComp Search
8 models · 16,896 tasks · last run 2026-08-18
Hard-to-locate facts on the live web, scored on persistent multi-step research.
| Rank | Model | Accuracy | Avg cost / task |
|---|---|---|---|
| 1 | Perplexity: Sonar Deep Research | 64.2% | $0.86 |
| 2 | Google: Gemini 3 Deep Research | 61.7% | $0.74 |
| 3 | OpenAI: GPT-5.2 Pro | 57.3% | $1.51 |
| 4 | Anthropic: Claude Opus 5 | 52.8% | $0.62 |
| 5 | Google: Gemini 3 Pro | 48.1% | $0.28 |
| 6 | OpenAI: GPT-5.2 | 45.2% | $0.41 |
DeepSearchQA Search
9 models · 8,940 tasks · last run 2026-08-20
Questions whose answers are lists, scored for exhaustive retrieval with no padding.
| Rank | Model | Accuracy | Avg cost / task |
|---|---|---|---|
| 1 | Google: Gemini 3 Deep Research | 70.8% | $0.69 |
| 2 | Perplexity: Sonar Deep Research | 68.4% | $0.81 |
| 3 | OpenAI: GPT-5.2 | 57.1% | $0.43 |
| 4 | Anthropic: Claude Opus 5 | 54.9% | $0.58 |
| 5 | xAI: Grok 4.2 | 51.2% | $0.24 |
| 6 | Google: Gemini 3 Pro | 49.7% | $0.27 |
WideSearch Search
7 models · 4,112 tasks · last run 2026-08-25
Fill an entire table; answer-item accuracy scores partial matches.
| Rank | Model | Accuracy | Avg cost / task |
|---|---|---|---|
| 1 | Google: Gemini 3 Deep Research | 66.1% | $0.77 |
| 2 | Perplexity: Sonar Deep Research | 63.9% | $0.90 |
| 3 | OpenAI: GPT-5.2 Pro | 59.8% | $1.48 |
| 4 | Anthropic: Claude Opus 5 | 54.4% | $0.60 |
| 5 | OpenAI: GPT-5.2 | 50.2% | $0.40 |
Sample data for demonstration — results are illustrative, not live evaluations.
Every model above is routable through LokaRouter at the listed per-token prices.