Most enterprise teams choose an AI model the same way: they check a leaderboard, pick a name near the top, and connect it to CRM, ERP, and warehouse data. The results that come back often look different from what the ranking promised, and that gap is why AI model benchmarks on enterprise data matter more than any public score.
That gap runs through all of Foundations 2026, CData’s November 5 event on enterprise AI, where we’ll look at the AI gateway from every angle: the case for it, how it works, how the context engine learns your business, and what a real rollout looks like.
In one of the Foundations sessions, Amit Naik, our VP of AI Architecture, and I will present “CData Labs: Testing the Gateway in the Open,” where we put the gateway’s claims to the test. About 10 years ago, I wrote CData’s JDBC performance harnesses, and the questions then were throughput and heap usage. Agents changed the questions. In the session, we’ll walk through what we tested, what came back, and where to get the harnesses so you can test your own scenarios.
Why leaderboard rankings don’t predict enterprise results
Amit frames the problem in two parts. First, public benchmark questions leak into training data over time, so a rising score can reflect memorization as much as capability (Zhou et al.; Deng et al., NAACL 2024). Second, leaderboards average performance across many tasks into a single number, and that number says little about the specific work your teams need a model to do. One vendor-authored preprint found that traditional benchmark rankings poorly predict agentic performance on business tasks (arXiv). The models themselves are good, but what they can reach, and how they reach it, is the variable no leaderboard measures.
What CData Labs provides
CData Labs is CData’s open research program that benchmarks AI models and MCP implementations on live enterprise data through CData Connect AI, and publishes the harnesses, runners, and raw results so anyone can reproduce them. The lab runs as a weekly working session with engineering and product. Every test starts with a written hypothesis and a prediction, and results get published whether they favor CData or not. The lab maintains its own benchmark repos, built to rerun against every new model release, alongside the product team’s open skills repo for Connect AI users. So far, the research has followed three tracks: accuracy, token efficiency, and model behavior across gateway configurations.
What we’ve already published
In March, we published our accuracy research, where Connect AI reached 98.5% accuracy on enterprise questions while other MCP implementations ranged from 65% to 75%. In June, we looked at token efficiency: the same question consumed 183,000 tokens with open access to raw data and 4,500 tokens once context and guardrails were configured in Connect AI. That result depends on a one-time developer setup, and the cost of that setup is real, but it pays back on every agent call that follows. In September, we published a model benchmark covering 22 models from eight model developers across 1,034 runs on live CRM, warehouse, and ITSM data. I’ll hold those findings for the session.
The questions we’ll answer on November 5
The September paper raised as many questions as it answered, and the platform has changed underneath it since. These are the questions we’ll put on screen:
If two models both get a cross-source question 100% right, how far apart can the bill be?
What happens when you hand 22 models a write task against live systems, and what does failure look like when it happens?
Where does most of your token bill actually come from, and is it the model?
What changes in the results now that the context engine and gateway routing are live in the platform?
Once the gateway equalizes accuracy and safety, how should you choose a model?
Where does CData Labs go next?
Amit and I will walk through the scenarios and the results, then show you where to run them yourself.
Watch AI models tested on live enterprise data
Register for Foundations 2026 to spend November 5 with the people building the AI gateway and the teams already running on it. CEO Amit Sharma and CPTO Raviv Levi open the day with where enterprise AI adoption needs the most help, Dr. Nick Golovin walks through the context engine, and Anaqua CIO Erik Bailey shares what his rollout looked like. Bring your questions about model choice, token cost, and write safety to the Labs session, then visit CData Labs, grab the harness, and check our work.
Frequently asked questions
When is the CData Labs session at Foundations 2026?
Foundations 2026 runs November 5, 2026. “CData Labs: Testing the Gateway in the Open,” with Jerod Johnson and Amit Naik, is one of the event’s sessions. Registration is free.
Can I reproduce CData Labs benchmarks myself?
Yes. CData Labs publishes the harnesses, benchmark runners, and raw results. The Connect AI Model Benchmark repo on GitHub includes the task prompts, tool schemas, scoring code, and a golden-answer generator, so you can point the harness at your own data or use the provided sample data.
Why not just use public LLM leaderboards to pick a model?
Leaderboards blend many tasks into one score, and their questions can leak into training data. Neither tells you how a model performs against your systems through your data layer.
