Since our beginning, CData has done one critical thing better than anyone: connect to enterprise systems and understand what’s actually inside them. Every object, every relationship, every validation rule, every custom field no vendor knows exists. That work became the data layer embedded in Palantir, Google, and hundreds of enterprise products, and the connectivity more than 10,000 companies run in production today.
For years this has been important. With enterprise AI, this data layer is now non-negotiable.
Token Cost, AI Governance, & Agent Accuracy
As people and agents work deeper and broader across the business, token spend is climbing and agents are being asked to do far more than summarize. IT leaders are dealing with three questions:
What is all of this costing?
What can agents see and do?
Can we trust the answers and actions?
The AI gateway market emerged to address these challenges, but most gateways so far only offer point solutions. Some govern model traffic. Others broker tool calls. What none of them start with is the connection to the systems where the work happens. It’s that connection that decides accuracy, efficiency, and trust downstream: whether the gateway knows what the data means, who is entitled to it, what an agent changed, and what the answer should have been.
With company context built in, an AI gateway generates real return: token spend drops because requests stop carrying more context than needed, answers hold steady because definitions live in one place, and the highest-value work of agents taking action across systems finally has the guardrails to go live.
Introducing Connect AI Gateway
Connect AI Gateway is a single connectivity and control point for everything AI touches in the enterprise. Through it, people and agents reach the models you approve, and those models reach the systems where work happens with live access, applied context, and permissions and policies enforced at the source. IT sets token budgets, rate limits, and routing rules by user, team, or workflow, and spend rolls up to the same ledger from each one.
It runs on hundreds of CData-built, schema-aware connectors, each carrying the data models, relationships, authentication models, and operations of the system it reaches. Alongside them, the Gateway governs any LLM and MCP server you bring, whether vendor-shipped or custom-built, under the same policy and observability. Agents get one governed path to every system they touch.
Connect AI Gateway in action: The life of a prompt
The best way to describe the value gained across the business is with a real-life, seemingly simple prompt. Someone in finance asks a question: what enterprise revenue is at risk in Q3?
Most AI stacks answer this poorly, regardless of the model chosen. The reason: three words in this sentence have no fixed meaning.
“Enterprise revenue” is a definition unique to your company, and most companies have more than one field across different systems that looks like it.
“Q3” depends on when your fiscal year starts.
“At risk” isn’t a field anywhere; it lives in support tickets and in the judgment of the people who read them.
A model left to guess will pick or create plausible definitions, and nobody downstream will catch the mistakes. Connect AI Gateway processes the prompt before it reaches a model.
Meaning is settled before anything runs
The Gateway reads the prompt against your context graph. “Enterprise” and “this quarter” resolve against system context. “Revenue” resolves against the semantic definitions your team already maintains. “At risk” resolves against the support tickets and procedures where that judgment actually lives. Settling all three before a single tool call fires is what ensures the final answer is correct, and this can only be done from the data layer.
The task picks the model, not the other way around
The Gateway classifies what the request actually needs. Recognizing this prompt is analytical rather than a lookup, it routes to a reasoning model, but not necessarily the largest one on the market. Once the question has been resolved in the context layer, a lighter model reaches the same answer as a frontier model at a fraction of the cost. Most gateways route on latency and price alone, which is a guess about difficulty.
Only the right tools open, under the right permissions
The Gateway binds the tools this task needs across Salesforce, NetSuite, and ServiceNow, and leaves the rest of the catalog closed. The agent is never shown the full tool list, so it doesn’t spend tokens reasoning about it. Each connection then carries its own policy, as well as universal policies set centrally, checked per request and resolved to the person or agent who asked:
Salesforce: read and write, on selected tables and fields only
NetSuite: read-only, scoped to one report
ServiceNow: read-only, with personally identifiable information (PII) masked on the way out
The work happens in the Gateway, not the model
Salesforce joins NetSuite on your own enterprise definition, and the at-risk flags from ServiceNow append as a column. What travels to the model is one resolved table rather than a pile of raw records. In the CData Labs token benchmark, that difference was 183,541 tokens and 22 tool calls against 4,427 tokens and one. Models are an expensive place to perform a join, and a slow one.
The answer returns through a final policy check and lands in a single audit trail: the prompt, the identity behind it, the records touched, the policy that allowed them, and the response that went back. Security teams can finally feel confident green lighting the higher value agentic use cases that have sat on shelves for months.
Learnings compound. The entire interaction encodes back into the context graph via agentic memory. The next person to ask about at-risk enterprise revenue gets that resolution immediately, and so does every other agent and model you route through the Gateway.
A deeper look: How the Gateway works
With Connect AI Gateway, every request from a person or agent passes through the gateway on its way to a model and into your systems, carrying identity, context, and policy the whole trip.
Enterprise MCP, built in, not tacked on
Connect a system and agents can work in it the same day. Access hundreds of CData-built MCP servers including hard-to-reach sources like SAP, NetSuite, and Workday, as well as the warehouses, databases, and on-premises and legacy systems organizations run on. Scoped tools expose only what a task requires. One managed platform with authentication, permissions, and auditing replaces the server-per-system sprawl and the weeks of hardening each server takes.
Connect to hundreds of first party CData MCP servers on day one, plus register any 3rd party MCP server
Semantic context that compounds
Agents answer consistently because every prompt and tool call is grounded in what the connectors already know, layered with your documented knowledge: metric definitions, business terminology, and the semantic models you already maintain. The Gateway sits where intent meets execution. It sees the prompt that asked how ARR is tracking against plan and the work that answered it, down to the fields, joins, and calculation, and it keeps the two together. Approved corrections hold the same way: tell it once that bookings exclude professional services, and every request after that applies it.
Import, build, and reinforce semantic context to attach meaning to every prompt across your business
Governance at every step a user or agent takes
Agents act with exactly the access their users have, and nothing more. Policy applies at the model, covering who can call what with rate and spend limits. It applies at the tool, covering which systems, what level of access, and when an action needs human approval. And it applies at the data, where every request carries two identities: the person it serves and the agent acting for them. Write actions are validated against the source system’s own business rules before they commit. One audit trail traces any answer back to the user and agent behind it, the prompt, the model, the tool, the records returned, and the policy that allowed it.
Inherit, establish, and enforce policies down to the agent, the data source, the tool, and the field
Reduce costs at every layer where tokens accrue
Most gateways can tell you what AI costs, but only Connect AI drives those costs down. It optimizes for the most efficient token usage before anything runs: schema-carrying tools make one precise call and return the answer set instead of the record set; agents load the tools a task needs instead of the full catalog; grounded context cuts the retries that come from guessing what data means; and routing sends a simple lookup to an inexpensive model, reserving frontier models for complex multi-system analyses. Metering is table stakes, and it is all there, with spend attributed and budgeted by team, agent, model, and tool.
Set budgets, routing rules, and overage policies by user, team, and workload
Portable context, and no platform to steer you toward
The Gateway has no warehouse to fill, no model to favor, and no platform to pull you deeper into. Context is normalized into a graph and stored outside any single model or data platform, so whichever model answers a request inherits the same definitions, relationships, and know-how. If a provider goes down, requests fail over with your context intact. Where privacy or cost calls for it, route to a local model. Switch anything tomorrow, and your context arrives with you.
All imported, built, and learned knowledge is mapped in a viewable, editable context graph
What the numbers show
We recently launched CData Labs to provide the benchmarks behind these claims. This includes the harnesses that produced them, so you can run them against your own setup.
Architecture drives accuracy more than model choice. Across 378 prompts spanning CRM, project management, warehouse, and ERP systems, Connect AI returned 98.5% accuracy against 65–75% for other ways of exposing the same data. Small per-step gaps compound fast: at 75% per step, fewer than a quarter of five-step workflows finish correctly.
Precise context costs less than large context. The same multi-source question, asked through Connect AI instead of raw table access, dropped from 183,541 tokens and 22 tool calls to 4,427 tokens and one call. That’s 97.6% fewer tokens for the same answer.
With the right context, the model matters less than the invoice suggests. Across 1,034 benchmark runs, 17 of 22 models returned the identical correct answer on live CRM, warehouse, and ITSM data. Only the price changed, and it changed by 178x.
When the context is precise, an inexpensive model reaches the same answer as a frontier one, and the Gateway is what decides which one runs. See the full methodology and raw results at cdata.com/labs.
Put the Gateway to work
Connect AI Gateway brings your models, your agents, and your systems under one control point, with your own context applied to every request and your policies enforced down to the record. Connect one source, set your policies, and point your agents at the endpoint. You can see what the Gateway resolved before a single token reaches a model.
Connect AI Gateway is available now through an early access program. Request access, or join us at Foundations on November 5 for the gateway from every angle.
