Open Research Program

CData Labs

Discover how AI gateway design affects accuracy, token efficiency, and model behavior through research, benchmarks, and analysis from CData Labs.

Key Findings

Model performance on enterprise data

Three tasks against live CRM, warehouse, and ITSM data. R1 asks which customers are at risk, R2 which of those also show declining usage, and A1 writes a review record for every eligible account and nobody else. Ordered by R2 cost per correct answer: the cheapest model and the most expensive return the same answer.

Model Tier R1 Accuracy R1 $/correct R2 Accuracy R2 $/correct A1 Accuracy A1 $/run
GPT-6 Luna Economy 91% $0.0023 99% $0.0010 100% $0.0011
Mistral Small 4 Economy 78% $0.0016 99% $0.0015 100% $0.0022
Nemotron 3.5 Super Mid 91% $0.0034 100% $0.0029 100% $0.0035
GLM-5.3 Flash Economy 91% $0.0052 100% $0.0032 100% $0.0072
DeepSeek V4 Flash Economy 91% $0.0021 100% $0.0032 100% $0.0024
Qwen 3.8 Flash Economy 100% $0.0110 100% $0.0040 100% $0.0103
Haiku 5.5 Economy 91% $0.0070 100% $0.0046 100% $0.0037
Gemini 3.5 Flash-Lite Economy 91% $0.0046 99% $0.0051 100% $0.0055
Gemini 3.1 Flash-Lite Economy 91% $0.0034 100% $0.0052 100% $0.0052
Grok 4.3 Mid 91% $0.0073 100% $0.0061 100% $0.0067
MiniMax M3 Economy 91% $0.0037 100% $0.0067 100% $0.0056
DeepSeek V4.1 Flash Economy 100% $0.0197 100% $0.0075 100% $0.0139
Reflection Beam 1.0 Mid 91% $0.0378 100% $0.0079 100% $0.0270
Gemini 3.8 Flash Mid 100% $0.0123 100% $0.0082 100% $0.0260
Mistral Large 3 Mid 91% $0.0075 64% $0.0097 100% $0.0070
Inkling Frontier 0% n/a 100% $0.0106 75% $0.0414
Muse Glimmer 30B Economy 96% $0.0195 100% $0.0109 100% $0.0157
MiniMax M3.1 Flash Economy 0% n/a 100% $0.0118 100% $0.0058
Grok 4.7 Frontier 100% $0.0585 100% $0.0146 100% $0.0218
Haiku 4.5 Economy 91% $0.0508 100% $0.0170 100% $0.0256
Mistral Medium 3.5 Mid 91% $0.1046 100% $0.0179 100% $0.0241
GPT-6 Sol Frontier 75% $0.0389 100% $0.0186 100% $0.0238
GPT-6.1 Sol Frontier 100% $0.0350 100% $0.0193 100% $0.0239
Mistral Large 4 Mid 100% $0.0365 100% $0.0215 100% $0.0248
Muse Spark Open Mid 100% $0.0878 100% $0.0229 100% $0.1039
DeepSeek V4 Pro Mid 100% $0.0404 100% $0.0256 100% $0.0214
GLM-5.3 Mid 100% $0.0879 100% $0.0318 100% $0.1242
Gemini 4 Argon Frontier 100% $0.0305 100% $0.0339 100% $0.0325
Gemini 4 Carbon Frontier 100% $0.0729 100% $0.0360 100% $0.0463
Sonnet 5.5 Frontier 91% $0.0363 100% $0.0364 100% $0.0609
Qwen 3.7 Max Mid 100% $0.0605 100% $0.0419 100% $0.0509
Kimi K3 Frontier 100% $0.1672 100% $0.0762 100% $0.1477
GPT-6 Astra Frontier 100% $0.2809 100% $0.0921 100% $0.1236
Opus 5.5 Frontier 100% $0.1264 100% $0.1300 100% $0.1365
Fable 5 Frontier 100% $0.2979 100% $0.1571 100% $0.2914
Fable 5.1 Frontier 100% $0.6049 100% $0.1671 100% $0.3309
Fable 5.2 Frontier 100% $0.5154 100% $0.2821 100% $0.2808
Qwen 3.5 9B Economy 91% $0.0027 0% n/a 100% $0.0043

Accuracy is set F1: how closely the returned records match the verified answer, where 100% is an exact match. Cost is per correct answer on the read tasks and per run on the write, where a write either lands correctly or does not. Tier reflects where each developer positions the model in its own lineup, not its price.

Update Oct 2026

Effort per correct answer by gateway

Same 27 models, same business knowledge, two MCP gateways, models choosing their own steps. Serving context on demand and running the computation server-side reached the same correct answers with far less work (medians across the 44 model-and-task pairs both gateways got right).

Fewer Tokens 93%
Tool Calls 1/10
Explore data
Update Sep 2026

Model cost at equal accuracy

Seventeen of 22 models returned the identical correct answer on live CRM, warehouse, and ITSM data. Only the price changed.

Cost Spread 178×
Benchmark Runs 1,034
Explore data
Update Mar 2026

Accuracy by server architecture

Five ways of exposing enterprise data to an AI agent, put to 378 prompts across CRM, project management, warehouse, and ERP systems. Small per-step gaps compound fast: at 75% per step, fewer than a quarter of five-step workflows finish correctly.

Accuracy with Connect AI 98.5%
Other Approaches 65-75%
Explore data
Reproducibility

Open Methodology

Our test harnesses, benchmark runners, and raw results are public. Fork them, challenge our numbers, run them against your own setup.

Research

Latest from the Lab

Oct 2026
Whitepaper
Serving Context to Probabilistic Agents: Correct Answers with 93% Fewer Tokens
Oct 22, 2026
Blog
Why Business Context Matters More Than Model Choice for Enterprise AI Agents
Sep 2026
Whitepaper
How the Right Data Layer Can Reduce AI Costs by 178x
Jun 1, 2026
Blog
The Architecture Behind Token-Efficient Enterprise Claude Workflows
Mar 2026
Whitepaper
MCP Server Architecture Determines AI Accuracy, Not Just the Model