Discover how a bimodal integration strategy can address the major data management challenges facing your organization today.
Get the Report →Build MVC Applications with Connectivity to Databricks Data
This article shows how to use only the Entity Framework and the CData ADO.NET provider to access Databricks from an ASP.NET MVC application.
In this article, we will guide you through the process of utilizing wizards within Visual Studio to seamlessly integrate the CData ADO.NET Provider for Databricks into a basic MVC (Model, View, Controller) project.
About Databricks Data Integration
Accessing and integrating live data from Databricks has never been easier with CData. Customers rely on CData connectivity to:
- Access all versions of Databricks from Runtime Versions 9.1 - 13.X to both the Pro and Classic Databricks SQL versions.
- Leave Databricks in their preferred environment thanks to compatibility with any hosting solution.
- Secure authenticate in a variety of ways, including personal access token, Azure Service Principal, and Azure AD.
- Upload data to Databricks using Databricks File System, Azure Blog Storage, and AWS S3 Storage.
While many customers are using CData's solutions to migrate data from different systems into their Databricks data lakehouse, several customers use our live connectivity solutions to federate connectivity between their databases and Databricks. These customers are using SQL Server Linked Servers or Polybase to get live access to Databricks from within their existing RDBMs.
Read more about common Databricks use-cases and how CData's solutions help solve data problems in our blog: What is Databricks Used For? 6 Use Cases.
Getting Started
Create the Entity Framework Model
Follow the steps below to save connection properties and map tables to entities in the data model.
- Create a new MVC project in Visual Studio. In this example, the project name is MvcDatabricksApp.
If you are using Entity Framework 6, you will need to take the preliminary step of registering the Databricks Entity Framework provider for your project. See the "LINQ and Entity Framework" chapter in the help documentation for a guide.
Note that MVC 3 scaffolding and MVC 4 scaffolding do not support Entity Framework 6. You can use your scaffolding with Entity Framework 6 by upgrading to the latest version of MVC.- To add the .edmx file from the designer, right-click your Models folder and click Add New Item. Select ADO.NET Entity Data Model, name the model, and click Add. In this example, the name of the model is DatabricksModel.
- In the Entity Data Model wizard, select the option 'EF Designer from database'. The Entity Data Model wizard is displayed.
- Click New Connection. Select CData Databricks Data Source in the dialog that is displayed.
Specify the required connection string properties.
To connect to a Databricks cluster, set the properties as described below.
Note: The needed values can be found in your Databricks instance by navigating to Clusters, and selecting the desired cluster, and selecting the JDBC/ODBC tab under Advanced Options.
- Server: Set to the Server Hostname of your Databricks cluster.
- HTTPPath: Set to the HTTP Path of your Databricks cluster.
- Token: Set to your personal access token (this value can be obtained by navigating to the User Settings page of your Databricks instance and selecting the Access Tokens tab).
A typical connection string is below:
Server=127.0.0.1;Port=443;TransportMode=HTTP;HTTPPath=MyHTTPPath;UseSSL=True;User=MyUser;Password=MyPassword;
Name the connection and select whether to include sensitive information, such as connection credentials, in the connection string. For simplicity, this example saves sensitive information in Web.config. The connection settings are saved as DatabricksEntities.
- Select the tables and views you need. In this example, Customers is imported. Also, the option to pluralize object names is deselected in this example. Click Finish to create the .edmx file.
- Build your project to complete this step.
Scaffold the Controller and Views
Once you've established the model and completed the project build, you can employ ASP.NET Scaffolding wizards to generate both the controller and the views.
- In Solution Explorer, right-click the controllers folder and click Add -> Controller. Select MVC 5 Controller with views, using Entity Framework.
- In the Add Controller dialog that is then displayed, select the following options:
- Model class: Select a table you imported; for example, Customers.
- Data context class: Select your context class.
-
Leave the default values for the other fields.