Ten years ago I had a front-row seat to a rebuild. Enterprises were tearing out the expensive private lines that connected their offices and moving that traffic onto the public internet, with a new piece of technology in the middle deciding where each packet went, what it could do and at what cost. The networking world called it SD-WAN (Software Defined Wide Area Network). I spent seven years at Cisco on different aspects of that shift, and the sequence was the same everywhere. First the conversation was about cost. Then security, because employees were using the technology in ways nobody had signed up for. Then the control layer in the middle became one of the biggest businesses in networking, because it let enterprises harness a new technology responsibly, safely and efficiently.
What made it possible was visibility, and visibility came from getting all the traffic to flow through one gateway. Once you could see who was going where and what it cost, you could decide what to allow, what to block, what to optimize and how to get it there faster. Control follows visibility. It always has.
I'm watching a similar movie these days. Replace network traffic with prompts going to AI models, and applications reaching out to the internet with AI agents reaching into your enterprise systems. The same three questions are arriving, in the same order: how can I optimize my costs, how can I do it safely, and who is in control.
Today we're announcing the Connect AI Gateway. Here is what it is, why it belongs to us to build, and where it goes next.
With our customers, every step
CData has never built in isolation. We've built alongside our customers. For decades that meant live connectivity into the systems where enterprise data actually lives: SAP, Workday, NetSuite, Salesforce, the warehouses, the databases, hundreds of sources. That connectivity became the layer other companies build on; it powers Salesforce, Google and Palantir products today.
When our customers started putting AI to work on that data, they asked for the same connectivity through MCP, so we built a managed MCP platform: one place for authentication, permissions and audit instead of a hand-built server for every system. When they told us they already ran MCP servers of their own, we brought those under the same governance. And recently, in nearly every conversation, the same three problems: AI costs rising faster than anyone budgeted for; answers not reliable enough to act on; and governance that lets AI search but not act.
Three problems, one missing layer. That's the gateway. Partnering with our customers through their AI transformation is not a go-to-market strategy for us. It's the culture and the mission. Anaqua is a good example: they built a control plane that gives their AI tools governed, per-user access to data across their business systems, and they pushed us hard on what "governed" has to mean.
What we're announcing
The Connect AI Gateway sits between your agents and employees on one side and your models and data on the other: Model gateway and MCP gateway in one place, with routing, guardrails, observability, a policy engine and identity carried end to end for people and agents. None of that is exotic on its own. What matters is what it's built on.
Right answers start at the connection
Connectivity is not a feature of an AI gateway. It's what determines whether the gateway can be trusted. We don't hand the model a pile of raw records and hope it does the math. We understand the schemas, relationships and validation rules of the source systems, compute the answer in our data layer, and return the final answer to the model.
CData Labs put a number on it: 378 real enterprise prompts across CRM, project management, data warehouse and ERP systems, through five MCP approaches. Through CData Connect AI, the answers were correct 98.5% of the time. The others landed between 65% and 75%. For an agent running a five-step process, that gap is decisive: at 75% per step, fewer than a quarter complete correctly. The whitepaper and test harness are public; run it on your own data.
Cheaper models, same answers, faster
The standard gateway pitch is "route easy prompts to cheaper models," and the standard objection is "and lose accuracy and safety doing it." CData Labs tested that too: 22 models, economy to frontier, across 1,034 runs on live CRM, data warehouse and IT service management data. When the data layer carried the business logic, an economy model returned the same correct answers as the priciest frontier model in the study (Fable 5) at 178x lower token cost. Without the data layer, the best model was right only 40% of the time. Control accuracy and safety at the data layer, and model choice becomes mostly a cost decision. And because the gateway returns the answer set instead of the full record set, the model reads 97.6% less context: fewer tokens, fewer retries, fewer round trips. The answer isn't just cheaper. It comes back faster.
Every request carries a name and leaves a trail
Agents act with exactly the access their users have, enforced inside the source system, down to the record: one delegated identity per request, the agent acting for a named user, not a shared service account nobody can trace. Writes are validated against the system's own business rules before they commit; in the same study, 20 of 22 models wrote exactly the correct records through governed tools, while on raw data the worst run wrote thousands of incorrect values. Every answer traces back to the prompt, model, tool, records and policy behind it, and spend is attributed by team, agent, model and tool, so you can fund AI work instead of capping it.
Context that compounds
Every prompt that runs through the gateway carries a little information about how your company describes its own business: which fields people mean when they say "revenue," how a customer in Salesforce relates to a customer in NetSuite. The context engine collects that, starting from what the live connection already knows, importing the semantic definitions you've already invested in, and adding the informal knowledge that lives in SOPs, chat threads and people's heads.
Two design choices matter more than any feature. Context is built over your live systems, not over a copy. And the memory lives in our layer, not in the model's, so what your agents learn travels with you to any model, any cloud, any data store. Every cloud provider, frontier lab and data-store vendor will gladly build your context for you, at the small price of locking you in for the rest of your life. Build it once and choose freely.
The next year
Not a personal prediction: CData is well along its own AI transformation, and these are the needs we see emerging, in our own use and our customers'. The team is already building toward them.
Routing gets smarter: today it's mostly people asking questions, next year it's agents running multi-step workflows, and routing will pick the model, tool and data path for each step on intent and context, not just cost. Governance gets more natural: policy as code is here, policy in plain language is coming. Context gets deeper and more shared, with judgment: some of what one team teaches the engine becomes organizational fact, and some stays with that team, on purpose. The ecosystem gets wider: we won't rebuild what the leaders in security, identity and observability already do well, so we're already working on security partnerships with market leaders and considering more.
And the gateway keeps learning from what runs through it. That's the part I'm most excited about and the least willing to overclaim. We'll show it working before we talk about it loudly.
A partner through the transformation
Every platform shift I've lived through came with a business imperative. We had to support mobile. We had to go SaaS. Now it's AI transformation, and if the plan is to wrap a few APIs in MCP and let the agents run around, that's not going to end well. What companies need is a partner deeply rooted in data who will help them through that journey in a responsible, accurate, efficient and secure way. That's what we've been doing for decades, one layer at a time, with and for our customers. The gateway is the next layer.
Thank you to Anaqua and every customer who pushed us here with real problems, and to the CData team that turned them into a product. If you're thinking about, starting, or in the middle of your AI transformation and wrestling with cost, accuracy, speed or control, I'd love to compare notes.
Request early access today at cdata.com/ai
