Connect to RabbitMQ Data in the Denodo Platform

Denodo Platform is a data virtualization product providing a single point of contact for enterprise database data. When paired with the CData API Driver for JDBC, Denodo users can work with live RabbitMQ data alongside other enterprise data sources. This article explains how to create a virtual data source for RabbitMQ in the Denodo Virtual DataPort Administrator.

With built-in optimized data processing, the CData JDBC Driver offers unmatched performance for interacting with live RabbitMQ data. When you issue complex SQL queries to RabbitMQ, the driver pushes supported SQL operations, like filters and aggregations, directly to RabbitMQ and utilizes the embedded SQL engine to process unsupported operations client-side (often SQL functions and JOIN operations). Its built-in dynamic metadata querying allows you to work with and analyze RabbitMQ data using native data types.

Create the RabbitMQ Virtual Port

To connect to live RabbitMQ data from Denodo, you need to copy the JDBC Driver JAR file to the external library directory for Denodo and create a new JDBC Data Source from the Virtual DataPort Administrator tool.

1. Download the CData API Driver for JDBC installer, unzip the package, and run the JAR file to install the driver.
2. Copy the JAR File (and license file if it exists) from the installation location (typically C:\Program Files\CData\CData API Driver for JDBC\lib\) to the Denodo external library directory (C:\Denodo\Denodo Platform\lib-external\jdbc-drivers\cdata-api-19).
3. Open the Denodo Virtual DataPort Administrator tool and navigate to the Server Explorer tab.
4. Right-click "admin" and select New -> Data source -> JDBC.
5. Configure the JDBC Connection:
   • Name: your choice, e.g.: api
   • Database adapter: Generic
   • Driver class path: C:\Denodo\Denodo Platform\lib-external\jdbc-drivers\cdata-api-19
   • Driver class: cdata.jdbc.api.APIDriver

   • Database URI: Set this to a JDBC URL using the necessary connection properties. For example,

     jdbc:api:Profile=C:\profiles\\RabbitMQ.apip;AuthScheme=Basic;URL=http://localhost:15672;User=guest;Password=guest;

     

     Information on creating the Database URI follows:

     Built-In Connection String Designer

     For assistance in constructing the JDBC URL, use the connection string designer built into the RabbitMQ JDBC Driver. Either double-click the JAR file or execute the jar file from the command-line.

     	java -jar cdata.jdbc.api.jar
     	

     Fill in the connection properties and copy the connection string to the clipboard.

     About RabbitMQ Management HTTP API

     RabbitMQ is an open-source message broker that supports multiple messaging protocols. The RabbitMQ Management HTTP API provides HTTP-based access to management and monitoring data for a RabbitMQ server. The API exposes information about virtual hosts, exchanges, queues, bindings, connections, channels, consumers, users, permissions, policies, and cluster-wide statistics.

     The Management plugin must be enabled on the RabbitMQ server for the HTTP API to be available. By default, the management interface listens on port 15672.

     Using Basic Authentication

     RabbitMQ Management HTTP API uses HTTP Basic authentication. You must supply the username and password of a RabbitMQ management user.

     To enable access to the management API:

     1. Ensure the RabbitMQ Management plugin is enabled on your server (rabbitmq-plugins enable rabbitmq_management).
     2. Use an existing management user or create one with the appropriate management tag (management, policymaker, monitoring, or administrator).
     3. Note the full base URL of your RabbitMQ Management HTTP API (e.g., http://localhost:15672).

     After configuring your RabbitMQ server, set the following connection properties to connect:

     • AuthScheme: Set this to Basic.
     • URL: Set this to the base URL of your RabbitMQ Management HTTP API (e.g., http://localhost:15672).
     • User: Set this to your RabbitMQ management username (e.g., guest).
     • Password: Set this to your RabbitMQ management password.

     Example connection string:

     Profile=C:\profiles\RabbitMQ.apip;AuthScheme=Basic;URL=http://localhost:15672;User=guest;Password=guest;
     

     Available Tables

     The RabbitMQ profile provides access to the following tables:

     • Overview - Cluster-wide statistics and information about the RabbitMQ node
     • Nodes - Information about individual nodes in the RabbitMQ cluster
     • NodeMemory - Detailed memory usage breakdown for a specific cluster node
     • Connections - List of all open AMQP connections to the broker
     • Channels - List of all open AMQP channels across all connections
     • Consumers - List of all consumers registered across all queues
     • Exchanges - List of exchanges declared across all virtual hosts
     • Queues - List of queues declared across all virtual hosts
     • Bindings - List of all bindings between exchanges and queues
     • VirtualHosts - List of virtual hosts configured on the broker
     • VhostPermissions - User permissions within a specific virtual host
     • Users - List of all RabbitMQ users
     • Permissions - Permission records for all users across all virtual hosts
     • TopicPermissions - Topic-level permission records for all users
     • Policies - List of policies applied to queues and exchanges in virtual hosts
     • OperatorPolicies - List of operator policies applied to queues in virtual hosts
     • Parameters - List of component parameters (e.g., federation, shovel) per virtual host
     • GlobalParameters - List of global parameters that apply across all virtual hosts
     • VhostLimits - Resource limits configured for specific virtual hosts
     • UserLimits - Resource limits configured for specific users
     • FeatureFlags - List of feature flags and their enabled/disabled state on the node
     • DeprecatedFeatures - List of deprecated features and their usage state
     • AuthAttempts - Authentication attempt statistics for the node
     • ClusterName - The name of the RabbitMQ cluster
     • WhoAmI - Information about the currently authenticated management user
     • ExchangeBindingsSource - Bindings for which a specific exchange is the source
     • ExchangeBindingsDestination - Bindings for which a specific exchange is the destination
     • QueueBindings - Bindings for a specific queue within a virtual host
     
6. Click the "Test connection" button to confirm the configuration and click Save.

View RabbitMQ Data in the VirtualPort Administrator Tool

After creating the data source, you can create a base view of RabbitMQ data for use in the Denodo Platform.

1. Click the "Create base view" button in the newly created VirtualPort (admin.API).
2. Expand the object tree and select the objects (tables) you wish to import.
3. Click the "Create selected" button to create views of the RabbitMQ data.
   Optional: Click "Create associations from foreign keys" to define relationships between the objects.
4. With the view(s) created, navigate to a table (cdata_api_authattempts) in the Server Explorer and double-click the selected table.
5. In the new tab, click "Execution panel" to open a query panel.
6. Customize the query in the "Execute" tab or use the default:

   SELECT * FROM cdata_api_authattempts CONTEXT ('i18n'='us_est', 'cache_wait_for_load'='true')
   

   
7. Click Execute to view the data.

With the base view created, you can now work with live RabbitMQ data like you would any other data source in Denodo Platform, for example, querying RabbitMQ in the Denodo Data Catalog.

Download a free, 30-day trial of the CData API Driver for JDBC and start working with your live RabbitMQ data in Denodo Platform. Reach out to our Support Team if you have any questions.