Creating Databricks-Connected Applications in Unqork

Ready to get started?

Learn more or sign up for a free trial:

CData Connect



You can use CData Connect to feed Databricks data to Unqork and build custom business apps.

Overview

You can use CData Connect to feed Databricks data to Unqork. From there you can build custom business applications in a visual, drag-and-drop interface. So, how does this process work?

Unqork is a completely visual, no-code application development platform that helps large enterprises build complex custom software. CData Connect exposes Databricks as a REST endpoint, which you can then query from Unqork with simple HTTP requests. This enables you to create your own custom application with Databricks data.

What You'll Learn

In this article, you'll learn how to:

  • Add a CData Connect user
  • Connect to Databricks from CData Connect
  • Add Databricks OData endpoints in CData Connect
  • Configure CORS (Cross-Origin Resource Sharing)
  • Register Databricks (via CData Connect) as an Unqork service
  • Create a simple Unqork application to display Databricks data

Configuring CData Connect

To work with live Databricks data in Unqork, you need to:

  • Provide user access to the new virtual database in CData Connect
  • Connect to Databricks from CData Connect
  • Create OData endpoints for Databricks data

Add a CData Connect User

First, create a User in CData Connect. This lets you connect to Databricks from Unqork through CData Connect.

  1. Click the Users tab in CData Connect
  2. Click Add
  3. Enter a username in the User field
  4. Enter a password in the Password field
  5. Check the Active checkbox
  6. Configure a User
  7. Click Save Changes and make note of the new user's Authtoken

Connect to Databricks from CData Connect

CData Connect uses a straightforward, point-and-click interface to connect to data sources and generate APIs.

  1. Open CData Connect and click Databases
  2. Select "Databricks" from Available Data Sources
  3. Enter your authentication properties to connect to Databricks
    NOTE:

    To connect to a Databricks cluster, set the properties as described below.

    Note: The needed values can be found in your Databricks instance by navigating to Clusters, and selecting the desired cluster, and selecting the JDBC/ODBC tab under Advanced Options.

    • Server: Set to the Server Hostname of your Databricks cluster.
    • HTTPPath: Set to the HTTP Path of your Databricks cluster.
    • Token: Set to your personal access token (this value can be obtained by navigating to the User Settings page of your Databricks instance and selecting the Access Tokens tab).
  4. Click Test Database
  5. Click Privileges
  6. Click Add to add a new or existing user with the appropriate permissions
  7. Click Save Changes.

Add Databricks OData Endpoints in CData Connect

After connecting to Databricks, create OData Endpoints for the desired table(s)

  1. Click the OData tab in CData Connect
  2. Click Tables
  3. Click Add Tables
  4. Select the Databricks database
  5. Select the table(s) you wish to work with and click Next
  6. Save the settings

NOTE: If you want to edit the table definition (select specific fields and more), click on the created tables on the OData tab.

(Optional) Configure Cross-Origin Resource Sharing (CORS)

When accessing and connecting to multiple different domains from an application such as Unqork, there is a possibility of violating the limitations of cross-site scripting. In that case, configure the CORS settings:

  1. Click the OData tab in CData Connect
  2. Click Settings and configure the CORS settings as follows:
    • Enable cross-origin resource sharing (CORS): ON (checked)
    • Allow all domains without '*': ON (checked)
    • Access-Control-Allow-Methods: GET, PUT, POST, OPTIONS
    • Access-Control-Allow-Headers: Authorization
  3. Save the changes

Sample URLs for OData Feeds

Once you create a CData Connect user, connect to Databricks, and create OData endpoints, you can access OData feeds for Databricks data. The output of the previous steps will be a list of tables and the associated URLs. Below, you will see the URLs to access tables.

For information on accessing the tables, click the API tab on the top right of the CData Connect website.

For the URLs, you will need the URL of the CData Connect instance, likely in the form: https://www.cdatacloud.net/myinstance/. Since we are working with React, we will append the @json parameter to the end of URLs that do not return JSON data by default.

Table         URL
Entity (table) List https://www.cdatacloud.net/myinstance/api.rsc/
Metadata for table Customers http://www.cdatacloud.net/myinstance/api.rsc/Databricks_Customers/$metadata?@json
Customers http://www.cdatacloud.net/myinstance/api.rsc/Databricks_Customers

As with standard OData feeds, if you wish to limit the fields returned, you can add a $select parameter to the query, along with other standard OData URL parameters, such as $filter, $orderby, $skip, and $top. See the CData help documentation for more information on supported OData queries.

Create a Simple Application for Databricks Data

With the REST API configured for Databricks through CData Connect, you are ready to create a Databricks-connected application in Unqork. Here are the basic steps to this process:

  • Register Databricks (via CData Connect) as an Unqork service
  • Create a workspace in Unqork
  • Create a module-type application

Register Databricks (via CData Connect) as an Unqork Service

To connect to CData Connect, you need to give your environment permission to connect with an external service. Do this through Services Administration.

  1. Click the Settings drop-down at the top right corner of the Unqork Designer Platform.
  2. Click Administration.
  3. Under Integration, click Services Administration.
  4. Enter a Service Title, for example, CData Connect.
  5. Enter a Service Name, for example, cdataConnect.
  6. Enter a Service protocol + host. This is the API endpoint you want to reference. For example, http://XXX.ap-northeast-1.compute.amazonaws.com:8153/api.rsc/.
  7. Select Basic Auth from the Type of Authentications drop-down.
  8. Enter the CData Connect user ID in the User Name field.
  9. Enter the CData Connect user Authtoken in the Password field.
  10. Click Add Service.

Create a Workspace

Unqork lets you create your own workspace, which is a virtual container for you and your team to store and organize your applications. Once you build a workspace, you can add applications to it.

To create a workspace:

  1. Click Create Workspace
  2. Enter a unique Workspace Name
  3. Click Create

Your workspace tile appears under the Workspaces tab.

Create a Module-Type Application

With your workspace added, you are ready to create your application. You have the choice between a module-type and workflow-type application. For this use case, make a module-type application.

  1. Navigate to the Apps tab from your open workspace.
  2. Click Create App.
  3. Enter a unique App Name.
  4. Select a preview style from the Style drop-down. A style specifies your application's appearance to your end-users. If left blank, your application uses the environment's default style.
  5. Select Module from the App Type drop-down.
  6. Select New Entrypoint Module from the App Entrypoint Module drop-down.
  7. Click Create.

You will land on the Modules tab of your new application. It should look like the following:

Click the title to open the module in the Module Editor.

To complete your application, you need the following components:

  • Hidden component: stores the data retrieved by the Plug-In
  • Initializer component: executes the Plug-In when displaying tables
  • Plug-In component: retrieves Databricks data through CData Connect and stores it in the Hidden component
  • ViewGrid component: creates an HTML table from the data stored in the Hidden component

Configure the Hidden Component

This Hidden component stores the Databricks data retrieved through CData Connect.

  1. Drag and drop a Hidden component onto your canvas.
  2. Enter a Property ID and Canvas Label Text.

    NOTE: Property IDs must be in Camel Case (stylized as camelCase). For example, hiddenComponent. Since the Hidden component is not visible to end-users, use your Property ID as the Canvas Label Text.

  3. Click Save.

Configure the Plug-in Component

Next, add a Plug-In component to request data from CData Connect. This component connects with the service you set up in Services Administration.

  1. Drag and drop a Plug-In component onto your canvas, placing it below the Hidden component.
  2. Enter a Property ID and Canvas Label Text.

    NOTE: You must use Camel Case (stylized as camelCase). For naming best practices, start your Plug-In's Property ID with plugin. For example, pluginGetAWProducts.

  3. Configure the Outputs table as follows:
    • Enter the Property ID of your Hidden component in the Property ID column.
    • Enter value in the Mapping column.
  4. Select External as the Service Type
  5. From the External Services drop-down, select your CData Connect service. This is the external service you set up in Services Administration.
  6. Add the specific endpoint for your CData Connect resource to the end of the Data Source URL. For example, AdventureWorks2012_Production_Product.

    NOTE: Your Data Source URL should have a value similar to: https://www.cdata.../myinstce/api.rsc/AdventureWorks2012_Production_Product. Check the API tab in CData Connect.

  7. Click Save.

Configure the Initializer Component

The Initializer component is what fires the Plug-In to retrieve your Databricks data.

  1. Drag and drop an Initializer component onto your canvas, placing it above the Plug-In component.
  2. Enter a Property ID and Canvas Label Text.

    NOTE: You must use Camel Case (stylized as camelCase). For naming best practices, start your Initializer's Property Name with init. For example, initGetAWProducts.

  3. Set New Submission as the Trigger Type.
  4. In the Outputs table, enter the following:
    • Enter the Property ID of your Plug-IN in the Property ID column's field.
    • Enter trigger in the Type column's field.
    • Enter GO in the Value column's field.
  5. Click Save.

Configure the ViewGrid Component

The ViewGrid component references the data stored in your Hidden component. Then, it displays the Databricks data to your end-users in a dashboard (or table).

  1. Drag and drop a ViewGrid component onto your canvas, placing it below the Plug-In component.
  2. Enter a Label and Property Name.

    NOTE: You must use Camel Case (stylized as camelCase). For naming best practices, start your ViewGrid's Property Name with vg. For example, vgAWProductView.

  3. In the Inputs table, set id to the Property ID of the hidden component.
  4. In the Display table:
    • In the id column: enter the IDs of the fields you want to display.
    • In the heading column: enter the title you want for each field. This is how the field will display to your end-users.
  5. Click Save.
  6. Save your module.

Once you add and configure the above components, you can preview how your application looks to end-users. In the Module Editor, click Preview Module to see the front-facing view (or "Express View") of the module.

Your Express View module should look something like this:

More Information & Free Trial

With CData Connect and Unqork, you can easily build Databricks-connected applications. Request a free trial of CData Connect and start working with Databricks data in Unqork today.