We are proud to share our inclusion in the 2024 Gartner Magic Quadrant for Data Integration Tools. We believe this recognition reflects the differentiated business outcomes CData delivers to our customers.
Get the Report →How to Access BigQuery Data Using Entity Framework
This article shows how to access BigQuery data using an Entity Framework code-first approach. Entity Framework 6 is available in .NET 4.5 and above.
Microsoft Entity Framework serves as an object-relational mapping framework for working with data represented as objects. Although Visual Studio offers the ADO.NET Entity Data Model wizard to automatically generate the Entity Model, this model-first approach may present challenges when your data source undergoes changes or when you require greater control over entity operations. In this article, we will delve into the code-first approach for accessing BigQuery data through the CData ADO.NET Provider, providing you with more flexibility and control.
About BigQuery Data Integration
CData simplifies access and integration of live Google BigQuery data. Our customers leverage CData connectivity to:
- Simplify access to BigQuery with broad out-of-the-box support for authentication schemes, including OAuth, OAuth JWT, and GCP Instance.
- Enhance data workflows with Bi-directional data access between BigQuery and other applications.
- Perform key BigQuery actions like starting, retrieving, and canceling jobs; deleting tables; or insert job loads through SQL stored procedures.
Most CData customers are using Google BigQuery as their data warehouse and so use CData solutions to migrate business data from separate sources into BigQuery for comprehensive analytics. Other customers use our connectivity to analyze and report on their Google BigQuery data, with many customers using both solutions.
For more details on how CData enhances your Google BigQuery experience, check out our blog post: https://www.cdata.com/blog/what-is-bigquery
Getting Started
- Open Visual Studio and create a new Windows Form Application. This article uses a C# project with .NET 4.5.
- Run the command 'Install-Package EntityFramework' in the Package Manger Console in Visual Studio to install the latest release of Entity Framework.
Modify the App.config file in the project to add a reference to the BigQuery Entity Framework 6 assembly and the connection string.
Google uses the OAuth authentication standard. To access Google APIs on behalf of individual users, you can use the embedded credentials or you can register your own OAuth app.
OAuth also enables you to use a service account to connect on behalf of users in a Google Apps domain. To authenticate with a service account, you will need to register an application to obtain the OAuth JWT values.
In addition to the OAuth values, you will need to specify the DatasetId and ProjectId. See the "Getting Started" chapter of the help documentation for a guide to using OAuth.
<configuration> ... <connectionStrings> <add name="GoogleBigQueryContext" connectionString="Offline=False;DataSetId=MyDataSetId;ProjectId=MyProjectId;InitiateOAuth=GETANDREFRESH" providerName="System.Data.CData.GoogleBigQuery" /> </connectionStrings> <entityFramework> <providers> ... <provider invariantName="System.Data.CData.GoogleBigQuery" type="System.Data.CData.GoogleBigQuery.GoogleBigQueryProviderServices, System.Data.CData.GoogleBigQuery.Entities.EF6" /> </providers> <entityFramework> </configuration> </code>
- Add a reference to System.Data.CData.GoogleBigQuery.Entities.EF6.dll, located in the lib -> 4.0 subfolder in the installation directory.
- Build the project at this point to ensure everything is working correctly. Once that's done, you can start coding using Entity Framework.
- Add a new .cs file to the project and add a class to it. This will be your database context, and it will extend the DbContext class. In the example, this class is named GoogleBigQueryContext. The following code example overrides the OnModelCreating method to make the following changes:
- Remove PluralizingTableNameConvention from the ModelBuilder Conventions.
- Remove requests to the MigrationHistory table.
using System.Data.Entity; using System.Data.Entity.Infrastructure; using System.Data.Entity.ModelConfiguration.Conventions; class GoogleBigQueryContext : DbContext { public GoogleBigQueryContext() { } protected override void OnModelCreating(DbModelBuilder modelBuilder) { // To remove the requests to the Migration History table Database.SetInitializer<GoogleBigQueryContext>(null); // To remove the plural names modelBuilder.Conventions.Remove<PluralizingTableNameConvention>(); } }
- Create another .cs file and name it after the BigQuery entity you are retrieving, for example, Orders. In this file, define both the Entity and the Entity Configuration, which will resemble the example below:
using System.Data.Entity.ModelConfiguration; using System.ComponentModel.DataAnnotations.Schema; [System.ComponentModel.DataAnnotations.Schema.Table("Orders")] public class Orders { [System.ComponentModel.DataAnnotations.Key] public System.String OrderName { get; set; } public System.String Freight { get; set; } }
- Now that you have created an entity, add the entity to your context class:
public DbSet<Orders> Orders { set; get; }
- With the context and entity finished, you are now ready to query the data in a separate class. For example:
GoogleBigQueryContext context = new GoogleBigQueryContext(); context.Configuration.UseDatabaseNullSemantics = true; var query = from line in context.Orders select line;