CData Labs
Discover how AI gateway design affects accuracy, token efficiency, and model behavior through research, benchmarks, and analysis from CData Labs.
Model performance on enterprise data
Three tasks against live CRM, warehouse, and ITSM data. R1 asks which customers are at risk, R2 which of those also show declining usage, and A1 writes a review record for every eligible account and nobody else. Ordered by R2 cost per correct answer: the cheapest model and the most expensive return the same answer.
| Model | Tier | R1 Accuracy | R1 $/correct | R2 Accuracy | R2 $/correct | A1 Accuracy | A1 $/run |
|---|---|---|---|---|---|---|---|
| GPT-6 Luna | Economy | 87% | $0.0009 | 99% | $0.0010 | 100% | $0.0011 |
| Mistral Small 4 | Economy | 91% | $0.0014 | 99% | $0.0015 | 100% | $0.0022 |
| Nemotron 3.5 Super | Mid | 91% | $0.0043 | 99% | $0.0029 | 100% | $0.0035 |
| DeepSeek V4 Flash | Economy | 91% | $0.0021 | 100% | $0.0032 | 100% | $0.0024 |
| GLM-5.3 Flash | Economy | 95% | $0.0085 | 75% | $0.0043 | 100% | $0.0072 |
| Haiku 5.5 | Economy | 87% | $0.0076 | 100% | $0.0046 | 100% | $0.0037 |
| Gemini 3.5 Flash-Lite | Economy | 91% | $0.0040 | 99% | $0.0051 | 100% | $0.0055 |
| Gemini 3.1 Flash-Lite | Economy | 91% | $0.0030 | 100% | $0.0052 | 100% | $0.0052 |
| Qwen 3.8 Flash | Economy | 95% | $0.0115 | 75% | $0.0053 | 100% | $0.0103 |
| Grok 4.3 | Mid | 91% | $0.0040 | 100% | $0.0061 | 100% | $0.0067 |
| MiniMax M3 | Economy | 91% | $0.0037 | 100% | $0.0067 | 100% | $0.0056 |
| DeepSeek V4.1 Flash | Economy | 95% | $0.0187 | 100% | $0.0075 | 100% | $0.0139 |
| Reflection Beam 1.0 | Mid | 91% | $0.0615 | 100% | $0.0079 | 100% | $0.0270 |
| Gemini 3.8 Flash | Mid | 100% | $0.0128 | 100% | $0.0082 | 100% | $0.0260 |
| Mistral Large 3 | Mid | 91% | $0.0044 | 64% | $0.0097 | 100% | $0.0070 |
| Inkling | Frontier | 91% | $0.0875 | 100% | $0.0106 | 75% | $0.0414 |
| Muse Glimmer 30B | Economy | 100% | $0.0188 | 100% | $0.0109 | 100% | $0.0157 |
| Grok 4.7 | Frontier | 93% | $0.0358 | 100% | $0.0146 | 100% | $0.0218 |
| Haiku 4.5 | Economy | 91% | $0.0313 | 100% | $0.0170 | 100% | $0.0256 |
| GPT-6 Sol | Frontier | 75% | $0.0427 | 100% | $0.0186 | 100% | $0.0238 |
| GPT-6.1 Sol | Frontier | 98% | $0.0338 | 100% | $0.0193 | 100% | $0.0239 |
| Mistral Large 4 | Mid | 95% | $0.0541 | 100% | $0.0215 | 100% | $0.0248 |
| Muse Spark Open | Mid | 95% | $0.1165 | 100% | $0.0229 | 100% | $0.1039 |
| DeepSeek V4 Pro | Mid | 100% | $0.0519 | 100% | $0.0256 | 100% | $0.0214 |
| GLM-5.3 | Mid | 95% | $0.1541 | 100% | $0.0318 | 100% | $0.1242 |
| Gemini 4 Argon | Frontier | 100% | $0.0327 | 100% | $0.0339 | 100% | $0.0325 |
| Sonnet 5.5 | Frontier | 91% | $0.0373 | 100% | $0.0364 | 100% | $0.0609 |
| Qwen 3.7 Max | Mid | 98% | $0.0628 | 100% | $0.0419 | 100% | $0.0509 |
| Kimi K3 | Frontier | 95% | $0.1926 | 100% | $0.0762 | 100% | $0.1477 |
| GPT-6 Astra | Frontier | 99% | $0.1700 | 100% | $0.0921 | 100% | $0.1236 |
| Opus 5.5 | Frontier | 100% | $0.1067 | 100% | $0.1300 | 100% | $0.1365 |
| Fable 5 | Frontier | 91% | $0.2436 | 100% | $0.1571 | 100% | $0.2914 |
| Fable 5.1 | Frontier | 98% | $0.3471 | 100% | $0.1671 | 100% | $0.3309 |
| Qwen 3.5 9B | Economy | 0% | — | 0% | — | 100% | $0.0043 |
Accuracy is set F1: how closely the returned records match the verified answer, where 100% is an exact match. Cost is per correct answer on the read tasks and per run on the write, where a write either lands correctly or does not. Tier reflects where each developer positions the model in its own lineup, not its price.
Model cost at equal accuracy
Seventeen of 22 models returned the identical correct answer on live CRM, warehouse, and ITSM data. Only the price changed. Based on internal testing by CData Software (Q3 2026). No independent third-party verification. Actual cost gaps varied among models, testing conducted using sandbox accounts containing known data sets that mirror production account structures. Results may not be representative of performance in live production environments, and results may vary. Organizations should conduct their own independent testing before making purchasing or implementation decisions.
Tokens per answered question
The same multi-source question, asked through CData Connect AI instead of raw table access, dropped from 183,541 tokens and 22 tool calls to 4,427 tokens and one. Based on internal testing by CData Software (Q2 2026). No independent third-party verification. Actual token gaps varied among configurations, testing conducted using sandbox accounts containing known data sets that mirror production account structures. Results may not be representative of performance in live production environments, and results may vary. Organizations should conduct their own independent testing before making purchasing or implementation decisions.
Accuracy by server architecture
Five ways of exposing enterprise data to an AI agent, put to 378 prompts across CRM, project management, warehouse, and ERP systems. Small per-step gaps compound fast: at 75% per step, fewer than a quarter of five-step workflows finish correctly. Based on internal testing by CData Software (Q4 2025). No independent third-party verification. Actual accuracy gaps varied among platforms and MCP approaches, testing conducted using sandbox accounts containing known data sets that mirror production account structures. Results may not be representative of performance in live production environments, and results may vary. Organizations should conduct their own independent testing before making purchasing or implementation decisions. 75% range of average accuracy across platforms, results differ by MCP approach.
Open Methodology
Our test harnesses, benchmark runners, and raw results are public. Fork them, challenge our numbers, run them against your own setup.