Connect to Apache Airflow Data in JRuby

JRuby is a high-performance, stable, fully threaded Java implementation of the Ruby programming language. The CData JDBC Driver for Apache Airflow makes it easy to integrate connectivity to live Apache Airflow data in JRuby. This article shows how to create a simple JRuby app that connects to Apache Airflow data, executes a query, and displays the results.

Configure a JDBC Connection to Apache Airflow Data

Before creating the app, note the installation location for the JAR file for the JDBC Driver (typically C:\Program Files\CData\CData JDBC Driver for Apache Airflow\lib).

JRuby natively supports JDBC, so you can easily connect to Apache Airflow and execute SQL queries. Initialize the JDBC connection with the getConnection function of the java.sql.DriverManager class.

Start by setting the Profile connection property to the location of the ApacheAirflow Profile on disk (e.g. C:\profiles\ApacheAirflow.apip). Next, set the ProfileSettings connection property to the connection string for ApacheAirflow (see below).

ApacheAirflow API Profile Settings

Apache Airflow 3 uses JWT Bearer tokens for API authentication. You can generate a token from the Airflow web UI under Settings or via the Airflow CLI using airflow users create and the /api/v2/auth/token endpoint. Note that this profile targets Apache Airflow 3.x using the /api/v2 REST API. The legacy /api/v1 endpoint used by Airflow 2.x is not supported.

After setting the following connection properties, you are ready to connect:

• AuthScheme: Set this to APIKey.
• APIKey: Set this to your Apache Airflow JWT Bearer token.
• Server: Set this to the base URL of your Airflow instance (e.g. http://localhost:8080).

Built-in Connection String Designer

For assistance in constructing the JDBC URL, use the connection string designer built into the Apache Airflow JDBC Driver. Either double-click the JAR file or execute the jar file from the command-line.

java -jar cdata.jdbc.api.jar

Fill in the connection properties and copy the connection string to the clipboard.

Below is a typical JDBC connection string for Apache Airflow:

jdbc:api:Profile=C:\profiles\ApacheAirflow.apip;AuthScheme=APIKey;ProfileSettings='APIKey=your_jwt_token;Server=http://localhost:8080';

Create a JRuby App with Connectivity to Apache Airflow Data

Create a new Ruby file (for example: APISelect.rb) and open it in a text editor. Copy the following code into your file:

require 'java'
require 'rubygems'
require 'C:/Program Files/CData/CData JDBC Driver for Apache Airflow 2018/lib/cdata.jdbc.api.jar'

url = "jdbc:api:Profile=C:\profiles\ApacheAirflow.apip;AuthScheme=APIKey;ProfileSettings='APIKey=your_jwt_token;Server=http://localhost:8080';"
conn = java.sql.DriverManager.getConnection(url)
stmt = conn.createStatement
rs = stmt.executeQuery("SELECT DagRunId, State FROM DagRuns")
while (rs.next) do
  puts rs.getString(1) + ' ' + rs.getString(2)
end

With the file completed, you are ready to display your Apache Airflow data with JRuby. To do so, simply run your file from the command line:

jruby -S APISelect.rb

Writing SQL-92 queries to Apache Airflow allows you to quickly and easily incorporate Apache Airflow data into your own JRuby applications. Download a free trial today!