Discover how a bimodal integration strategy can address the major data management challenges facing your organization today.
Get the Report →
Query Kafka Data from Node.js
The API Server exposes Web services that allow connectivity to your data. Use the OData endpoint of the CData API Server to execute CRUD queries to Kafka data from Node.js.
The CData API Server, when paired with the ADO.NET Provider for Kafka, exposes Kafka data (or data from any of 200+ other ADO.NET Providers) as an OData endpoint, which can be queried from Node.js using simple HTTP requests. This article shows how to use the API Server to request JSON-formatted Kafka data in Node.js.
Set Up the API Server
Follow the steps below to begin producing secure Kafka OData services:
Deploy
The API Server runs on your own server. On Windows, you can deploy using the stand-alone server or IIS. On a Java servlet container, drop in the API Server WAR file. See the help documentation for more information and how-tos.
The API Server is also easy to deploy on Microsoft Azure, Amazon EC2, and Heroku.
Connect to Kafka
After you deploy the API Server and the ADO.NET Provider for Kafka, provide authentication values and other connection properties needed to connect to Kafka by clicking Settings -> Connections and adding a new connection in the API Server administration console.
Set BootstrapServers and the Topic properties to specify the address of your Apache Kafka server, as well as the topic you would like to interact with.
Authorization Mechanisms
- SASL Plain: The User and Password properties should be specified. AuthScheme should be set to 'Plain'.
- SASL SSL: The User and Password properties should be specified. AuthScheme should be set to 'Scram'. UseSSL should be set to true.
- SSL: The SSLCert and SSLCertPassword properties should be specified. UseSSL should be set to true.
- Kerberos: The User and Password properties should be specified. AuthScheme should be set to 'Kerberos'.
You may be required to trust the server certificate. In such cases, specify the TrustStorePath and the TrustStorePassword if necessary.
You can then choose the Kafka entities you want to allow the API Server access to by clicking Settings -> Resources.
Authorize API Server Users
After determining the OData services you want to produce, authorize users by clicking Settings -> Users. The API Server uses authtoken-based authentication and supports the major authentication schemes. Access can also be restricted based on IP address; all IP addresses except the local machine are restricted by default. You can authenticate as well as encrypt connections with SSL.
Consume Kafka OData Feeds from Node.js
OData feeds are easy to work with in Node.js. You can use the HTTP client in Node.js to request JSON-formatted data from the API Server's OData endpoint. After making the request, you can construct the body of the response and call the JSON.parse() function to parse it into records.
The code below will make an authenticated request for SampleTable_1 data. The example URL below applies a simple filter that searches for records with a value of 100 in the Column2 column.
var http = require('http');
http.get({
protocol: "http:",
hostname: "MyServer.com",
port: MyPort,
path: "/api.rsc/SampleTable_1?$filter=" + encodeURIComponent("Column2 eq '100'"),
auth: 'MyUser:MyAuthtoken'
},
function(res) {
var body = '';
res.on('data', function(chunk) {
body += chunk;
});
res.on('end', function() {
console.log(body);
var jsonData = JSON.parse(body);
});
}).on('error', function(e) {
console.log("Error: ", e);
});
Data Connectors
ETL/ ELT Solutions
Cloud & API Connectivity
OEM & Custom Drivers