Learn more
 

Ready to get started?

Download a free trial of the Azure Data Lake Storage Driver to get started:

 Download Now

Learn more:

Azure Data Lake Storage Icon Azure Data Lake Storage JDBC Driver

Rapidly create and deploy powerful Java applications that integrate with Azure Data Lake Storage.

In this article

Related articles

A PostgreSQL Interface for Azure Data Lake Storage Data


Use the Remoting features of the Azure Data Lake Storage ODBC Driver to create a PostgreSQL entry-point for data access.

There are a vast number of PostgreSQL clients available on the Internet. From standard Drivers to BI and Analytics tools, PostgreSQL is a popular interface for data access. Using the remoting features of our JDBC Drivers, you can now create PostgreSQL entry-points that you can connect to from any standard client.

To access Azure Data Lake Storage data as a PostgreSQL database, use the Remoting feature of the CData JDBC Driver for Azure Data Lake Storage and the MySQL foreign data wrapper (FDW) from EnterpriseDB. In this article, we install the FDW and query Azure Data Lake Storage data from PostgreSQL Server.

Configure the Connection to Azure Data Lake Storage

Follow the steps below to configure the driver's MySQL daemon to use the credentials and other connection properties needed to connect to Azure Data Lake Storage. The MySQL daemon exposes Azure Data Lake Storage data as a MySQL database named CDataADLS. Add connection properties to the databases section of the configuration file for the daemon. The configuration file for the daemon is located in the lib subfolder of the installation directory for the driver.

Below is a typical connection string:

[databases] azure data lake storage = "Schema=ADLSGen2;Account=myAccount;FileSystem=myFileSystem;AccessKey=myAccessKey;InitiateOAuth=GETANDREFRESH"

Additionally, create a user in the users section.

You can find all of the configuration options for the MySQL daemon in the help documentation.

Start the Remoting Service

Follow the steps below to enable the MySQL Remoting feature of the CData JDBC Driver for Azure Data Lake Storage.

  1. The driver creates a default configuration suitable for testing: Simply start the service to connect to Azure Data Lake Storage data.

  2. Start the MySQL Remoting Service with the following command: java -jar cdata.jdbc.adls.jar -f cdata.jdbc.adls.remoting.ini

Build and Install the MySQL Foreign Data Wrapper

The Foreign Data Wrapper can be installed as an extension to PostgreSQL, without recompiling PostgreSQL.

If pgxn is available for your operating system, you can install with the following:

pgxn install mysql_fdw USE_PGXS=1

Otherwise, follow the steps below to build it yourself:

  1. Install the MySQL C client library and obtain the source for the EnterpriseDB FDW for MySQL; from GitHub, for example.
  2. Build the FDW. Add the pg_config and mysql_config executables to your PATH: env PATH=/usr/local/pgsql/bin:/usr/local/mysql/bin:$PATH make USE_PGXS=1
  3. Install the FDW: make USE_PGXS=1 install

To complete the installation, you will need to load the libmysqlclient library into the environment; for example by adding it to the path.

Query Azure Data Lake Storage Data as a PostgreSQL Database

After you have installed the extension, follow the steps below to start executing queries to Azure Data Lake Storage data:

  1. Log into your database.
  2. Load the extension for the database: postgres=#CREATE EXTENSION mysql_fdw;
  3. Create a server object for Azure Data Lake Storage data: postgres=# CREATE SERVER ADLS FOREIGN DATA WRAPPER mysql_fdw OPTIONS (host '127.0.0.1', port '3309');
  4. Create a user mapping for the username and password of a user known to the MySQL daemon. postgres=# CREATE USER MAPPING for postgres SERVER ADLS OPTIONS (username 'admin', password 'test');
  5. Create the local schema: postgres=# CREATE SCHEMA ADLS_db;
  6. Import all the tables in the Azure Data Lake Storage database you defined in the daemon configuration file: postgres=# IMPORT FOREIGN SCHEMA "ADLS" FROM SERVER ADLS INTO ADLS_db;

You can now execute SELECT commands to Azure Data Lake Storage:

postgres=# SELECT * FROM ADLS_db."resources";