Excel Spreadsheet Automation on Apache Airflow Data with the QUERY Formula

The CData Excel Add-In for Apache Airflow provides formulas that can query Apache Airflow data. The following three steps show how you can automate the following task: Search Apache Airflow data for a user-specified value and then organize the results into an Excel spreadsheet.

The syntax of the CDATAQUERY formula is the following:

=CDATAQUERY(Query, [Connection], [Parameters], [ResultLocation]);

This formula requires three inputs:

• Query: The declaration of the Apache Airflow data records you want to retrieve, written in standard SQL.

• Connection: Either the connection name, such as APIConnection1, or a connection string. The connection string consists of the required properties for connecting to Apache Airflow data, separated by semicolons.

  Start by setting the Profile connection property to the location of the ApacheAirflow Profile on disk (e.g. C:\profiles\ApacheAirflow.apip). Next, set the ProfileSettings connection property to the connection string for ApacheAirflow (see below).

  ApacheAirflow API Profile Settings

  Apache Airflow 3 uses JWT Bearer tokens for API authentication. You can generate a token from the Airflow web UI under Settings or via the Airflow CLI using airflow users create and the /api/v2/auth/token endpoint. Note that this profile targets Apache Airflow 3.x using the /api/v2 REST API. The legacy /api/v1 endpoint used by Airflow 2.x is not supported.

  After setting the following connection properties, you are ready to connect:

  • AuthScheme: Set this to APIKey.
  • APIKey: Set this to your Apache Airflow JWT Bearer token.
  • Server: Set this to the base URL of your Airflow instance (e.g. http://localhost:8080).
• ResultLocation: The cell that the output of results should start from.

Pass Spreadsheet Cells as Inputs to the Query

The procedure below results in a spreadsheet that organizes all the formula inputs in the first column.

1. Define cells for the formula inputs. In addition to the connection inputs, add another input to define a criterion for a filter to be used to search Apache Airflow data, such as DagId.
2. In another cell, write the formula, referencing the cell values from the user input cells defined above. Single quotes are used to enclose values such as addresses that may contain spaces.

=CDATAQUERY("SELECT * FROM DagRuns WHERE DagId = '"&B6&"'","Profile="&B1&";AuthScheme="&B2&";ProfileSettings="&B3&";Server="&B4&";Provider=API",B7)


3. Change the filter to change the data.