Last week, we launched CData Connect AI Gateway and went inside the Gateway to follow one prompt from question to governed answer. This week, we're peeling back another layer to look inside the context engine that uniquely powers it.
The context engine is where an agent's prompt gets its meaning. It settles what definitions a question refers to, which systems hold the answer, and which model should do the work. In this article, we will cover how it builds your company context and how it impacts agent accuracy, token spend, and user trust. And coming up on November 5th during Foundations 2026, our own Nick Golovin, Mike Albritton, and Amit Naik will dive even deeper into the context engine and provide a live demo of these latest capabilities.
Context decides whether an agent gets it right or confidently wrong
Connecting an agent to enterprise systems is the easy half. You install a Model Context Protocol (MCP) server, point it at Salesforce, and within an afternoon a model can turn a prompt into an API call. But then someone asks the agent for annual recurring revenue (ARR) this quarter and it returns a number that's confidently, plausibly wrong. It found a column called revenue, applied a filter it invented, and used the calendar year instead of the fiscal one.
Context closes that gap in three ways:
Gives prompts meaning. Words like "enterprise," "at risk," and "this quarter" have no fixed definition until something settles them.
Reconciles definitions. The same term often means different things in Salesforce and NetSuite, and context records which definition applies where.
Limits the work the agents and model must do. When the right column, filter, and join are known ahead of time, a large language model (LLM) reads from a prepared table instead of deducing one from thousands of raw records.
More than schema
Most gateways define context narrowly as a schema, a list of tool descriptions, a set of agent skills, or a library of system prompts that can be pasted into various AI assistants. A schema may tell an agent what a field is called, but it leaves far more out: which column finance trusts, why Salesforce never reconciles to billing, which opportunity stage counts as closed, etc. That knowledge lives in an analyst's head or a Slack thread from the CFO last March.
Hand-written skills and prompts help, but they're maintained per assistant and start decaying the week after launch. Meanwhile, agents without deeper context scan hundreds of schemas on every turn, re-derive yesterday's join, and might still get the fiscal calendar wrong.
Connect AI incorporates three core types of company context
The Connect AI context engine provides an agent what it needs to use your data correctly, not just find it. From day one, it builds a context graph that spans both structured and unstructured sources and impacts every step of the prompt journey.
System context
System context describes what each source holds and how it behaves, including entity maps, relationship guides, query patterns, unsupported operations, and the quirks of each system. It ships with CData connectors for hundreds of systems, including the hardest to model ones like SAP, NetSuite, and Workday, and it comes from more than a decade of connector engineering. A new Zendesk or SQL Server connection contributes real context on day one instead of an empty catalog waiting for someone to fill it in.
Semantic meaning
Semantic meaning bridges the business terminology of the company and it's data household. This covers metric definitions, and the various filters that belong to a definition. It also includes the models your data teams build. For example, when a derived view defines the right data fields, revenue logic, and cross-system joins for "Q2 revenue by region", the context layer documents it so agents discover it for every related prompt thereafter.
Unstructured company knowledge
Unstructured knowledge covers wikis, PDFs, documentation, and the know-how that was never written down. Connect AI reads and indexes all of it so that the policy that governs a data point lives alongside it. There's no separate retrieval-augmented generation (RAG) pipeline to stand up and maintain.
For every concept across all three types, the context engine captures definitions, relationships, usage guidance, vocabulary, and provenance. Experts can be assigned to promote tacit knowledge to the context engine, and a surprising answer can be traced to who defined it and when, instead of argued about.
Four ways Connect AI assembles context
Starting on day one, the context engine builds a company's context layer through four routes:
Source systems. Connectors carry entity maps, relationship guides, and query patterns, loaded before the first prompt.
Existing semantic models. Definitions from dbt, Power BI, Snowflake, and other catalogs map in directly, and data owners can author new ones with the review they'd give a code change.
Wikis and documents. Connect AI parses documentation into context the same layer holds, and generates context for sources that have no metadata at all.
Prompts and feedback. The Gateway sits in the execution path, so every query that ran, every prompt a user rephrased, and every result someone corrected becomes signal.
All three types of context are mapped in a viewable, editable graph representing shared knowledge across the organization
The context graph continuously improves with use
These four routes meet in the Connect AI Gateway to feed one comprehensive graph. Each concept (such as a metric, an object, or a definition) is a node that gets linked to the concepts it depends on. ARR, for example, is derived from opportunities, which belong to accounts, which reconcile against invoices in billing. An agent answering an ARR question follows those links instead of guessing how the pieces fit.
You stay in control of it. Concepts are stored as readable files, so the definition an analyst reads is the one the model retrieves. Each team's workspace sees only the context it's authorized to use, and learned changes are treated as proposals rather than mandates. IT admins can choose which types of changes auto-apply and which need an owner's approval.
The graph grows with use. Every system you connect adds source context, and every prompt the Gateway learns from sharpens what's already there.
How the learning happens
A semantic layer beside the query path can only record what someone typed into it. Because Connect AI is positioned directly in the prompt execution path, it can link an intent to the prompt that actually worked and fold that back into the model:
Conversational signals. A user asks for ARR, gets the wrong cut, and corrects it. Connect AI links that intent to the query that finally worked and updates the definition behind it.
Query mining. Commonly queried entities, frequent join paths, and popular filters surface over time and sharpen retrieval.
Conversation capture. An agent connected to Slack or Teams can be set to watch designated channels and flag discussions about metric definitions or schema changes as candidates to merge, catching decisions made in a thread that were never officially recorded in an SOP.
Automated reconciliation. Connect AI periodically reviews the whole model for conflicts, redundancies, and gaps, then reconciles them as sources evolve.
Human review. When the engine hits something it can't resolve, like two definitions of the same metric, it asks the right person and folds the answer back in.
Corrections validated by designated users get promoted to shared knowledge at the team, department, or organizational level
Connect AI puts context to work on every prompt
Context drives value only when it shapes a request before the model acts. Connect AI applies it in three steps to limit unnecessary token consumption while delivering enterprise-mandated accuracy.
Classify the prompt and choose the model
First, the Gateway’s query engine determines what a request needs. A single-object lookup goes to a fast, inexpensive model. Multi-source financial reasoning goes to a frontier model. Most gateways route on latency and price alone, which can be helpful to control cost but ultimately is a guess about difficulty. With Connect AI, the same understanding that answers "what does ARR mean" also informs which model runs, with nothing for a developer to configure.
Break down the prompt and find the records
Next, the engine resolves the prompt into the systems, tables, and records it needs. Ask "What's our ARR this quarter?" and the prompt resolves to the governed ARR definition: right formula, right filter, right fiscal calendar. It answers from one concept, with no schema crawl and no guessing which of four revenue columns was meant. Filters and aggregations push down to the source, so only the answer set returns.
Now ask the follow-up: "Why doesn't that match the number in Salesforce?" The definition already knows it reconciles against billing. The agent follows one relationship out to invoices, using a join path validated in advance rather than inferred. That's a reconciliation in one extra hop. Retrieval returns the relevant concept and its relationships, not everything the agent might conceivably need.
Rectify differences in meaning across sources
Finally, the engine resolves the places where systems disagree. A term can resolve differently in Salesforce than in NetSuite, so the graph holds a global concept as the parent and source-specific concepts beneath it. Vocabulary mappings mean "bookings," "closed won," and "new ARR" all resolve to the right object, even when a prompt uses someone else's words. Definitions apply consistently across systems without flattening every source into a lowest-common-denominator model.
The gateway parses the prompt, directs the agent to the right data sets and tools, and rectifies meaning across sources to deliver the highest level of agent accuracy
The result: accurate agents, lower spend, confident users
CData Labs benchmarked five MCP server approaches against 378 real-world prompts spanning CRM, enterprise resource planning (ERP), project management, and data warehouse scenarios. The spread between best and worst is a spread between gateway architectures, not models.
More accurate answers and actions. Connect AI returned 98.5% accuracy against 65–75% for the other approaches, and 100% on ERP queries, where the source-native MCP server scored 0%. Small per-step gaps compound: at 75% accuracy per step, fewer than a quarter of five-step workflows finish correctly.
Cheaper models and fewer tokens. One multi-source query dropped from 183,541 tokens and 22 tool calls to 4,427 tokens and one call, which is 97.6% fewer tokens for the same answer. Across 1,034 runs, 17 of 22 models returned the identical correct answer regardless of model chosen, reflecting a token cost variance of 178x.
Consistent responses across people and models. The finance prompts in Claude, support workflows in Microsoft Copilot, and internal agents calling a proprietary model all read one definition of ARR, with no drift between teams that prefer different tools. A fix that an analyst makes in one AI assistant improves the layer that every other model reads.
With context in place, model tier becomes a cost decision rather than an accuracy decision. See the full methodology at cdata.com/labs.
Start with context that ships on day one
Getting started doesn't require defining your business first. Connect a source and its context arrives with the connector, so the first query is already informed. Derived views and authored definitions then raise accuracy on the metrics your business argues about most. They improve a working system and aren't a prerequisite for one.
Context also stays yours. It's stored in the Gateway, outside any single model or data platform, so Claude, GPT, Gemini, Microsoft Copilot, and a model you host yourself all read from the same graph. It imports definitions from the catalogs and semantic layers you already use, and it doesn't ask you to fill a warehouse first. Changing or adding a provider becomes a routing decision, and everything your organization has taught the engine stays where it is.
The context engine is part of Connect AI Gateway and available through an early access program. Request access to generate your first context layer and define the metrics your organization argues about most, or join us at Foundations on November 5 to see the Gateway from every angle.
