Import Typeform Data Using Azure Data Factory

Microsoft Azure Data Factory (ADF) is a completely managed, serverless data integration service. When combined with CData Connect AI, ADF enables immediate cloud-to-cloud access to Typeform data within data flows. This article outlines the process of connecting to Typeform through Connect AI and accessing Typeform data within ADF.

CData Connect AI offers a cloud-to-cloud interface tailored for Typeform, granting you the ability to access live data from Typeform data within Azure Data Factory without the need for data replication to a natively supported database. Equipped with optimized data processing capabilities by default, CData Connect AI seamlessly channels all supported SQL operations, including filters and JOINs, directly to Typeform. This harnesses server-side processing to expedite the retrieval of the desired Typeform data.

Configure Typeform Connectivity for ADF

Connectivity to Typeform from Azure Data Factory is made possible through CData Connect AI. To work with Typeform data from Azure Data Factory, we start by creating and configuring a Typeform connection.

CData Connect AI uses a straightforward, point-and-click interface to connect to data sources.

1. Log into Connect AI, click Sources, and then click Add Connection

2. Select "Typeform" from the Add Connection panel

3. Enter the necessary authentication properties to connect to Typeform.

   Start by setting the Profile connection property to the location of the TypeForm Profile on disk (e.g. C:\profiles\TypeForm.apip). Next, set the ProfileSettings connection property to the connection string for TypeForm (see below).

   TypeForm API Profile Settings

   Authentication to TypeForm uses the OAuth standard.

   To authenticate to TypeForm, you must first register and configure an OAuth application with TypeForm here: https://admin.typeform.com/account#/section/tokens. Your app will be assigned a client ID and a client secret which can be set in the connection string. More information on setting up an OAuth application can be found at https://developer.typeform.com/get-started/.

   Note that there are several different use scenarios which all require different redirect URIs:

   • CData Desktop Applications: CData desktop applications (Sync, API Server, ArcESB) accept OAuth tokens at /src/oauthCallback.rst. The host and port is the same as the default port used by the application. For example, if you use http://localhost:8019/ to access CData Sync then the redirect URI will be http://localhost:8019/src/oauthCallback.rst.
   • CData Cloud Applications: CData cloud applications are similar to their desktop counterparts. If you access Connect AI at https://1.2.3.4/ then you should use the redirect https://1.2.3.4/src/oauthCallback.rst.
   • Desktop Application: When using a desktop application, the URI https://localhost:33333 is recommended.
   • Web Application: When developing a web application using the driver, use your own URI here such as https://my-website.com/oauth.

   After setting the following connection properties, you are ready to connect:

   • AuthScheme: Set this to OAuth.
   • InitiateOAuth: Set this to GETANDREFRESH. You can use InitiateOAuth to manage the process to obtain the OAuthAccessToken.
   • OAuthClientId: Set this to the Client Id that is specified in your app settings.
   • OAuthClientSecret: Set this to Client Secret that is specified in your app settings.
   • CallbackURL: Set this to the Redirect URI you specified in your app settings.
   
4. Click Save & Test
5. Navigate to the Permissions tab in the Add Typeform Connection page and update the User-based permissions.

Add a Personal Access Token

When connecting to Connect AI through the REST API, the OData API, or the Virtual SQL Server, a Personal Access Token (PAT) is used to authenticate the connection to Connect AI. It is best practice to create a separate PAT for each service to maintain granularity of access.

1. Click on the Gear icon () at the top right of the Connect AI app to open the settings page.
2. On the Settings page, go to the Access Tokens section and click Create PAT.
3. Give the PAT a name and click Create.
4. The personal access token is only visible at creation, so be sure to copy it and store it securely for future use.

With the connection configured and a PAT generated, you are ready to connect to Typeform data from Azure Data Factory.

Access Live Typeform Data in Azure Data Factory

To establish a connection from Azure Data Factory to the CData Connect AI Virtual SQL Server API, follow these steps.

1. Login to Azure Data Factory.

2. If you have not yet created a Data Factory, Click New -> Dataset.

3. In the search bar, enter SQL Server and select it when it appears. On the following screen, enter a name for the server. In the Linked service field, select New.

4. Enter the connection settings.
   • Name - enter a name of your choice.
   • Server name - enter the Virtual SQL Server endpoint and port separated by a comma: tds.cdata.com,14333
   • Database name - enter the Connection Name of the CData Connect AI data source you want to connect to (for example, API1).
   • User Name - enter your CData Connect AI username. This is displayed in the top-right corner of the CData Connect AI interface. For example, [email protected].
   • Password - select Password (not Azure Key Vault) and enter the PAT you generated on the Settings page.
   • Click Create.

5. In Set properties, set the Name, choose the Linked service we just created, select a Table name from those available, and Import schema from connection/store. Click OK.

6. After creating the linked service, the following screen should appear:

7. Click preview data to see the imported Typeform table.
You can now use this dataset when creating data flows in Azure Data Factory.

Get CData Connect AI

To get live data access to hundreds of SaaS, Big Data, and NoSQL sources directly from your cloud applications, try CData Connect AI today!