CData and PrimaLabs: Better Together for Open-Weight Models

Agent workloads have changed the economics of enterprise AI. Many teams now want to run open-weight models with enterprise data for the routine steps in an agent run, like classifying a request, drafting a query, or summarizing a result, because those steps make up most of the calls and rarely need a frontier model. What those teams won’t give up is the governance and live data access that production agents need. CData and PrimaLabs are working together to close that gap. PrimaLabs runs fast serverless inference for open-weight models such as Kimi K3.1 Pro, GLM 5.3 Flash, MiMo-V2.6-Pro, and DeepSeek V4.1 Flash, and it is planned as a named model provider in CData Connect AI Gateway through bring your own model. In this article, we’ll cover what each side contributes, what we tested, and how you can try it yourself.

At a glance: Open-weight models are language models with published trained weights, so teams can choose which provider runs them. PrimaLabs, a serverless inference API for open-weight models, has the lowest cost per task on Kimi K3.1 Pro and the lowest time per task on MiMo-V2.6-Pro and Nemotron 3.5 Lightning in Artificial Analysis provider benchmarks as of November 9, 2026. CData Connect AI Gateway, available through an early access program that runs until December 31, 2026, routes any OpenAI-compatible provider through one governed endpoint with live, permissioned access to enterprise data.

Why open-weight models are moving into agent stacks

“Open-weight models are language models whose trained weights are published, so teams can run them on a provider of their choice and decide where their data goes” (N-iX). For years, these models trailed the frontier by a wide margin. That gap has narrowed, and more enterprise teams now evaluate open-weight models for production work.

Cost is what drives the shift. Routine, high-volume agent steps rarely need a frontier model, and paying frontier prices for every one of those calls becomes a tax on scale as agent usage grows across your organization (Gradient Flow). Open-weight models let your teams match the cost of a call to the difficulty of the task, and the same model is often available from several providers at different prices.

That said, cheaper inference only pays off under two conditions. The model has to be reliable on the specific task you give it, and the controls around the model have to exist before it touches production data. A low price per token doesn’t help much if the answers are wrong or nobody can account for what the agent asked and saw.

Why AI projects using open-weight models stall without governed enterprise data

Most open-weight pilots stall on the controls and connections around the model. Three gaps come up again and again: the model has no governed path to company data, there are no spend or access controls by team, and there is no audit trail of what was asked and what was answered. Each gap is manageable in a pilot, where one engineer owns the API key and watches the logs, but each becomes a blocker once the agent is in front of real users.

Running or renting an open-weight model doesn’t satisfy risk-management expectations on its own. Frameworks like the NIST AI Risk Management Framework emphasize oversight, traceability, and accountability across the AI lifecycle, and those expectations apply no matter which model sits behind the agent.

Consider an agent that answers sales questions. It needs an inexpensive model to handle the volume, and it needs permissioned access to live CRM data so each rep sees only the accounts they’re entitled to see. Solving the first need without the second produces a cheap agent that nobody trusts with the data that makes it useful.

What PrimaLabs brings

PrimaLabs provides a serverless inference API for open-source models, built for agentic workloads (PrimaLabs). The API is OpenAI-compatible and priced per token, so teams can point an existing client at PrimaLabs and compare results on their own traffic. PrimaLabs adapts the full serving stack to how each application runs, so a coding agent making many short calls over the same codebase and a high-volume document pipeline each get fast, cost-efficient responses, without any change to the API call.

PrimaLabs runs its serverless API with zero data retention: prompts and completions are processed in memory and are not written to logs, databases, or exports. Only request metadata, such as model, timestamps, and token counts, is kept for billing and abuse prevention, and prompt caching is isolated to each account (PrimaLabs data retention).  The inference that PrimaLabs runs is also based in the U.S.  That matters for teams sending business context through a third-party inference provider. PrimaLabs is a partner in the DOE Genesis Mission with national labs and NVIDIA, and it has the lowest cost per task on Kimi K3.1 Pro and the lowest time per task on MiMo-V2.6-Pro and Nemotron 3.5 Lightning in Artificial Analysis provider benchmarks as of November 9, 2026.

“Open models are now fast and capable enough to handle much of the work in an agent run. What teams keep asking us is how to connect them to their own business data without losing control of it, and CData has solved that. Together, we’re giving customers a governed path to run production agents on open models.”

— Chaitanya Hiremath, Co-Founder, President & COO, PrimaLabs

How Connect AI Gateway handles bring your own model

Bring your own model means any provider that exposes an OpenAI-compatible API can be routed through CData Connect AI Gateway using the provider’s own credentials, with governance applied to every call. At CData, we built the Gateway so that model choice becomes a policy decision that lives in one place. Moving an existing agent onto the Gateway is a base URL and API key change.

Once a provider is registered, the Gateway applies the same controls to every request:

1. One endpoint in front of every provider, with provider keys held in Connect AI and callers using revocable virtual keys

2. Per-team budgets with soft and hard limits, checked before any tokens are spent

3. Model allowlists that decide which models each team or agent can call

4. A full log of every prompt and completion, attributed to the person or agent that made it

The Gateway covers the model side. Connect AI also connects the agent to live enterprise data through Model Context Protocol (MCP), enforcing the requesting user’s permissions across hundreds of enterprise sources without building or maintaining data pipelines. Because queries, joins, and filters run in the data layer, the model receives only the result it needs, which keeps token counts down regardless of the model. PrimaLabs is planned as a named provider in the Connect AI model catalog. You can learn more about routing and spend controls on the LLM Gateway page, or see how the Gateway compares with other options in LiteLLM vs. CData Connect AI.

What we tested

We wanted to see how the pairing holds up on a realistic business question before writing about it. Using PrimaLabs partner credits, our team routed the question “Which accounts renew this month?” through Connect AI Gateway to GLM 5.3 Flash on PrimaLabs serverless inference. The agent queried Salesforce through Connect AI with the user’s permissions applied, and returned a correct answer using 43K input and 1.8K output tokens for under one cent ($0.007).

This is a single tested scenario, and the cost will vary with the model, the question, and the amount of data returned. We tested serverless models only. Here’s what we’d tell technical readers to watch for:

1. Setup is a custom provider today. You add PrimaLabs to the LLM Provider list in the Connect AI Gateway as a custom LLM provider, using your own PrimaLabs API key and base URL, then point a routing rule at a model. Use the model ID exactly as PrimaLabs lists it, prefix included (for example primalabs-ai/GLM-5.3-Flash).

2. Evaluate your own workload. MCP tool calling worked on every serverless model we tested. As with any provider, results vary by model, tools, and question, so run your real business questions through a candidate model and review the outputs before you roll it out.ed. As with any provider, results vary by model, tools, and question, so run your real business questions through a candidate model and review the outputs before you roll it out.

Matching the model to the job

The pattern is simple to describe and harder to operate without the right controls. Send routine, high-volume steps to a low-cost open-weight model, and keep frontier models for the steps that need hard reasoning. Our own benchmark data supports this approach. Across 1,034 runs on 22 models against live Salesforce, Snowflake, and ServiceNow data, 21 of 22 models returned identical results through Connect AI’s governed tools, and the cost of the same correct answer varied by up to 178x depending only on the model chosen (CData efficiency study). Once the data layer handles accuracy, model choice becomes mostly a question of cost and throughput, and Gateway routing rules and fallbacks keep that choice inside policy.

The inference provider matters as much as the model. The same open-weight model runs at different speeds and prices across providers, and agent runs resend the same system prompt and tool definitions on every call. PrimaLabs automatically caches repeated prompt prefixes for each account, and cache hits are billed at a lower rate than standard input, which brings down cost per answer across a full agent run. A simple split for routing rules: send high-volume steps like classification, extraction, and summarizing to Nemotron 3.5 Lightning, agentic Q&A over live data to GLM 5.3 Flash, and complex planning to MiMo-V2.6-Pro.

Before assigning a model to an agent task, evaluate it against your own workload:

• Accuracy on your questions, using real prompts and real data in place of public benchmarks

• Tool-calling reliability, since agents fail quietly when a model formats tool calls poorly

• Latency at the concurrency your agents will run

• Cost per answer, measured across a full agent run instead of per token

• Data-retention terms from the inference provider

Over time, this becomes a portfolio decision you revisit as models and prices change. For more on how teams put this into practice, see our use cases for controlling token costs and managing model routing.

“We’re happy to bring PrimaLabs’ fast inference for open-weight models to CData Connect AI Gateway. Our customers are starting to move inference workloads onto open-weight models, and this integration gives them a smooth way to make that transition.”

— Amit Naik, VP of AI Architecture, CData Software

Run open-weight models against live enterprise data with CData Connect AI Gateway

Bring your own model to Connect AI Gateway, point an agent at governed, live enterprise data, and see what a routine business question costs on an open-weight model. Request early access to Connect AI Gateway to get 14 days of full Connect AI access, with Gateway routing and budget controls at no cost through December 31, 2026.

Frequently asked questions