by Tomas Restrepo | December 07, 2018

CData Architecture: Supporting Multiple Technologies

In my previous post, I described how we use a number of different factors to evaluate the complexity of building a driver for a new data source. Today, I'd like to give you a glimpse at our driver architecture.

Let's start with a high-level overview:

At the top of the diagram above we have some of the technologies we support, referred to as the driver front-ends. These adapt our driver model to each different driver/technology interface.

In the middle, we have all our core driver code, which is divided into two big categories. The first one is the driver support services which implements functionality that is common across many drivers. We'll dive into some of these further in this post.

The second part of the driver core is the data access stack, which is a key part of our architecture. This presents a very useful extensibility point that allows us to extend the driver core by adding layers above/below throughout the execution of most queries.

Finally, at the bottom, we have all the provider-specific code, which includes implementations for some of the extensibility points in the layers above that attach the driver-specific code to the query execution pipeline. This includes a driver-specific implementation of the data access stack.

Front-End Technologies

The top part of the figure presented above is the driver front-end. This is how a given data source is exposed to a specific data access technology, depending on the platforms we want to support. Each front-end technology is implemented in two parts:

  • Core implementation shared by all drivers - This contains all the driver-agnostic code, and is implemented as an internal framework on which driver-specific code plugs in. For example, our JDBC implementation includes base implementations of core JDBC concepts such as Statement or ResultSet.
  • Driver-specific implementation of the interface - This part is entirely done through automatic code generation, and is extremely flexible.

This last part is a key ingredient of our technology stack, that allows us to very quickly support new editions or new technologies in our drivers.

Here's a brief overview of how this works:

  • For each driver, we define key metadata using a declarative language. This includes basic things such as:
    • Driver name, description, and so on.
    • All the supported connection properties and associated documentation.
    • Editions that are supported by this driver.
  • From this metadata, we generate at build time:
    • All driver-specific code for each supported edition.
    • All HTML documentation for the product.
    • Installers and installation scripts

This layer gives us tremendous flexibility and simplifies our development process. Let’s examine how.

For each driver, we can choose which editions we want to support. For example, it might not make sense to publish a BizTalk adapter for a given data source, so we might choose to publish only traditional data access providers for it. A more common scenario, however, is making limited builds during driver development and while it is published as a beta release.

We can easily define common configuration options or connection properties once, and reuse them across many different drivers. This makes it easy to build a consistent interface and common behavior across all of them. For example, almost every single one of our drivers has connection-level properties for configuring firewall and proxy server properties. We define these just once, in a central location, ensuring that the way these are defined and documented is consistent across all drivers.

If we want to introduce a new connection property or option in a driver, we can declare it once and have it available across all supported editions and technologies without extra work. At the same time, we can still define edition-specific options, or hide them as necessary to simplify the configuration. When creating a new driver, the technology-specific frameworks and the provider-specific code are connected through well-defined, common interfaces, which makes it very simple to offer a provider in one of our supported technologies. If there is a bug in our technology-specific implementation, or a new feature we want offer, we can do it once by adjusting our code generation templates and resolve it across our entire driver lineup.

This model also makes us uniquely positioned to target new platforms or offer drivers across new technologies in record time, as we can build the necessary support code and templates only once, and leverage them across our entire data source portfolio.

Conclusion

In this initial post in the series, we've covered the high-level architecture of our drivers.

Our framework, based around code generation on the front-end, allows us to build native drivers that support different driver/adapter technologies, through a flexible and extensible model makes it very easy to support new ones.

This architecture allows us to create drivers that function consistently across technologies, with updates and improvements being supported all across the board.

In the next post in the series, we'll cover some of the core driver services in our framework.

Next: "Core Driver Services"  »