Import Cvent Data Using Azure Data Factory

Microsoft Azure Data Factory (ADF) is a completely managed, serverless data integration service. When combined with CData Connect AI, ADF enables immediate cloud-to-cloud access to Cvent data within data flows. This article outlines the process of connecting to Cvent through Connect AI and accessing Cvent data within ADF.

CData Connect AI offers a cloud-to-cloud interface tailored for Cvent, granting you the ability to access live data from Cvent data within Azure Data Factory without the need for data replication to a natively supported database. Equipped with optimized data processing capabilities by default, CData Connect AI seamlessly channels all supported SQL operations, including filters and JOINs, directly to Cvent. This harnesses server-side processing to expedite the retrieval of the desired Cvent data.

Configure Cvent Connectivity for ADF

Connectivity to Cvent from Azure Data Factory is made possible through CData Connect AI. To work with Cvent data from Azure Data Factory, we start by creating and configuring a Cvent connection.

CData Connect AI uses a straightforward, point-and-click interface to connect to data sources.

1. Log into Connect AI, click Sources, and then click Add Connection

2. Select "Cvent" from the Add Connection panel

3. Enter the necessary authentication properties to connect to Cvent.

   Before you can authenticate to Cvent, you must create a workspace and an OAuth application.

   Creating a Workspace

   To create a workspace:

   1. Sign into Cvent and navigate to App Switcher (the blue button in the upper right corner of the page) >> Admin.
   2. In the Admin menu, navigate to Integrations >> REST API.
   3. A new tab launches for Developer Management. Click on Manage API Access in the new tab.
   4. Create a Workspace and name it. Select the scopes you would like your developers to have access to. Scopes control what data domains the developer can access.
      • Choose All to allow developers to choose any scope, and any future scopes added to the REST API.
      • Choose Custom to limit the scopes developers can choose for their OAuth apps to selected scopes. To access all tables exposed by the driver, you need to set the following scopes:

        event/attendees:read	event/attendees:write	event/contacts:read
        event/contacts:write	event/custom-fields:read	event/custom-fields:write
        event/events:read	event/events:write	event/sessions:delete
        event/sessions:read	event/sessions:write	event/speakers:delete
        event/speakers:read	event/speakers:write	budget/budget-items:read
        budget/budget-items:write	exhibitor/exhibitors:read	exhibitor/exhibitors:write
        survey/surveys:read	survey/surveys:write

   Creating an OAuth Application

   After you have set up a Workspace and invited them, developers can sign up and create a custom OAuth app. See the Creating a Custom OAuth Application section in the Help documentation for more information.

   Connecting to Cvent

   After creating an OAuth application, set the following connection properties to connect to Cvent:

   • InitiateOAuth: GETANDREFRESH. Used to automatically get and refresh the OAuthAccessToken.
   • OAuthClientId: The Client ID associated with the OAuth application. You can find this on the Applications page in the Cvent Developer Portal.
   • OAuthClientSecret: The Client secret associated with the OAuth application. You can find this on the Applications page in the Cvent Developer Portal.
   
4. Click Save & Test
5. Navigate to the Permissions tab in the Add Cvent Connection page and update the User-based permissions.

Add a Personal Access Token

When connecting to Connect AI through the REST API, the OData API, or the Virtual SQL Server, a Personal Access Token (PAT) is used to authenticate the connection to Connect AI. It is best practice to create a separate PAT for each service to maintain granularity of access.

1. Click on the Gear icon () at the top right of the Connect AI app to open the settings page.
2. On the Settings page, go to the Access Tokens section and click Create PAT.
3. Give the PAT a name and click Create.
4. The personal access token is only visible at creation, so be sure to copy it and store it securely for future use.

With the connection configured and a PAT generated, you are ready to connect to Cvent data from Azure Data Factory.

Access Live Cvent Data in Azure Data Factory

To establish a connection from Azure Data Factory to the CData Connect AI Virtual SQL Server API, follow these steps.

1. Login to Azure Data Factory.

2. If you have not yet created a Data Factory, Click New -> Dataset.

3. In the search bar, enter SQL Server and select it when it appears. On the following screen, enter a name for the server. In the Linked service field, select New.

4. Enter the connection settings.
   • Name - enter a name of your choice.
   • Server name - enter the Virtual SQL Server endpoint and port separated by a comma: tds.cdata.com,14333
   • Database name - enter the Connection Name of the CData Connect AI data source you want to connect to (for example, Cvent1).
   • User Name - enter your CData Connect AI username. This is displayed in the top-right corner of the CData Connect AI interface. For example, [email protected].
   • Password - select Password (not Azure Key Vault) and enter the PAT you generated on the Settings page.
   • Click Create.

5. In Set properties, set the Name, choose the Linked service we just created, select a Table name from those available, and Import schema from connection/store. Click OK.

6. After creating the linked service, the following screen should appear:

7. Click preview data to see the imported Cvent table.
You can now use this dataset when creating data flows in Azure Data Factory.

Get CData Connect AI

To get live data access to 300+ SaaS, Big Data, and NoSQL sources directly from your cloud applications, try CData Connect AI today!