Use Databricks Agent Bricks to Talk to Your Elasticsearch Data via CData Connect AI

Databricks Agent Bricks is a fully managed, production-ready framework within the Databricks platform for building, deploying, and orchestrating AI agents. Agent Bricks enables teams to compose Supervisor Agents that coordinate external tools and data sources, including MCP Servers, to automate complex enterprise workflows with live, real-time data.

CData Connect AI offers a dedicated cloud-to-cloud interface for connecting to Elasticsearch data. Databricks Agent Bricks can authenticate securely against the CData Connect AI MCP Server without requiring user interaction. This allows Supervisor Agents to query and act on your Elasticsearch data autonomously, all without the need for data replication to a natively supported database.

In this guide, you will set up an OAuth App in CData Connect AI, configure an MCP Server connection in Databricks, and create a Supervisor Agent in Agent Bricks that queries live Elasticsearch data through the Connect AI MCP Server.

Getting Started

---

Prerequisites

Before you begin, ensure you have the following:

1. A Databricks workspace with access to the Mosaic AI — Agent Bricks feature.
2. A CData Connect AI account. Sign up for a free trial if you don’t have one.
3. At least one data source connected to your CData Connect AI account.

Step 1: Create an OAuth App in CData Connect AI

Databricks Agent Bricks uses OAuth Machine-to-Machine (M2M) authentication to connect to the CData Connect AI MCP Server. This requires creating a service-based OAuth App in your CData Connect AI account to generate a Client ID and Client Secret.

1. Click the Gear icon () in the top-right corner of Connect AI to open Settings.
2. Navigate to OAuth Apps and click + Create App. The Create OAuth App dialog appears.
3. Enter the following settings:
   • Name — Enter a descriptive name for the OAuth app (e.g., DatabricksOAuth).
   • Authentication Flow — Select Service-based (client credentials).
   • Callback URL — Since there is no callback URL required for M2M, enter a placeholder URL such as https://localhost/callback.
   • User — Select the CData Connect AI user account under which this OAuth app will run and access your Elasticsearch data.
   
4. Click Confirm. CData Connect AI creates the OAuth App and generates a Client ID and Client Secret.
5. Copy both the Client ID and Client Secret values. You will need them in the next step.

Step 2: Create an MCP Server Connection in Databricks

With your OAuth App credentials in hand, you can now create an HTTP connection in Databricks Unity Catalog that points to the CData Connect AI MCP Server and authenticates using OAuth M2M.

1. Open your Databricks workspace and navigate to the Catalog tab.
2. Click the + icon and select Create connection from the drop-down list.
3. In Step 1: Connection basics, enter the following:
   • Connection name — Enter a user-defined name for this connection (e.g., CData Connect AI MCP).
   • Connection type — Select HTTP.
   • Auth type — Select OAuth Machine to Machine.
   
4. Click Next.
5. In Step 2: Authentication, enter the following:
   • Host — Enter https://mcp.cloud.cdata.com.
   • Port — Leave the default port.
   • Client Secret — Enter the Client Secret you copied from the OAuth App.
   • Client ID — Enter the Client ID you copied from the OAuth App.
   • OAuth scope — Leave blank.
   
6. Click Next.
7. In Step 3: Connection details, enter the following:
   • Token endpoint — Enter https://cloud-login.cdata.com/oauth/token.
   • Base path — Enter /mcp.
   • Is MCP connection — Select (enable) this option.
   
8. Click Create connection. Databricks creates and validates the MCP Server connection.

Step 3: Create a Supervisor Agent in Agent Bricks

With the MCP Server connection established in Databricks, you can now create a Supervisor Agent in Agent Bricks that uses CData Connect AI as a tool to access your live Elasticsearch data.

1. In your Databricks workspace, navigate to the Agents tab.
2. Click Create Agent.
3. Select Supervisor Agent as the agent type.
4. In the Create Supervisor Agent dialog, enter the following:
   • Name — Enter a name for your agent.
   • Description — Enter a description of what this agent does.
5. Under Configure Agents, add a tool with the following settings:
   • Type — Select External MCP Server.
   • Unity Catalog connection — Select the MCP Server connection you created in the previous step.
   • Describe the content — Enter a description of what this MCP connection provides.
   
6. Click Create Agent. Agent Bricks provisions your Supervisor Agent.

Step 4: Explore Live Elasticsearch Data with Agent Bricks

Your Supervisor Agent is now configured and ready to query live Elasticsearch data through the CData Connect AI MCP Server. Use the Agent Bricks Playground to test natural language queries against your Elasticsearch data.

1. On your agent’s page, click Open in Playground.
2. Enter a natural language prompt to query your Elasticsearch data. Refer to CData’s prompt library for prompt ideas for your data source.

Get CData Connect AI

To get live data access to hundreds of SaaS, Big Data, and NoSQL sources directly from your cloud applications, try CData Connect AI today!