Query Kafka Data as a MySQL Database in Node.js

Ready to get started?

Learn more or sign up for a free trial:

CData Connect



Execute MySQL queries against Kafka data from Node.js.

You can use CData Connect Cloud to query Kafka data through a MySQL interface. Follow the procedure below to create a virtual database for Kafka in Connect Cloud and start querying using Node.js.

CData Connect Cloud provides a pure MySQL, cloud-to-cloud interface for Kafka, allowing you to easily query live Kafka data in Node.js — without replicating the data to a natively supported database. As you query data in Node.js, CData Connect Cloud pushes all supported SQL operations (filters, JOINs, etc) directly to Kafka, leveraging server-side processing to quickly return Kafka data.

Create a Virtual MySQL Database for Kafka Data

CData Connect Cloud uses a straightforward, point-and-click interface to connect to data sources and generate APIs.

  1. Login to Connect Cloud and click Databases.
  2. Select "Kafka" from Available Data Sources.
  3. Enter the necessary authentication properties to connect to Kafka.

    Set BootstrapServers and the Topic properties to specify the address of your Apache Kafka server, as well as the topic you would like to interact with.

    Authorization Mechanisms

    • SASL Plain: The User and Password properties should be specified. AuthScheme should be set to 'Plain'.
    • SASL SSL: The User and Password properties should be specified. AuthScheme should be set to 'Scram'. UseSSL should be set to true.
    • SSL: The SSLCert and SSLCertPassword properties should be specified. UseSSL should be set to true.
    • Kerberos: The User and Password properties should be specified. AuthScheme should be set to 'Kerberos'.

    You may be required to trust the server certificate. In such cases, specify the TrustStorePath and the TrustStorePassword if necessary.

  4. Click Test Database.
  5. Click Privileges -> Add and add the new user (or an existing user) with the appropriate permissions.

With the virtual database created, you are ready to connect to Kafka data from any MySQL client.

Query Kafka from Node.js

The following example shows how to define a connection and execute queries to Kafka with the MySQL module. You will need the following information:

  • Host name, or address, and port: The address of your instance of the Connect Cloud (myinstance.cdatacloud.net) and the port (3306)
  • Username and password: The username and password of a user you authorized on Connect Cloud
  • Database name: The database you configured for Kafka (kafkadb)

Connect to Kafka data and start executing queries with the code below:

var mysql      = require('mysql');
var fs         = require('fs');
var connection = mysql.createConnection({
  host     : 'myinstance.cdatacloud.net',
  database : 'kafkadb',
  port	   : '3306',
  user     : 'admin',
  password : 'password',
  ssl      : {
    ca : fs.readFileSync('C:/certs/myCA.pem')
  }
});
connection.connect();
connection.query('SELECT * FROM SampleTable_1', function(err, rows, fields) {
  if (err) throw err;
  console.log(rows);
});

connection.end();