Validate Google Cloud Storage Data with QuerySurge

QuerySurge is a smart data testing solution that automates data validation and testing. When paired with the CData JDBC Driver for Google Cloud Storage, QuerySurge can work with live Google Cloud Storage data. This article walks through connecting to Google Cloud Storage data from QuerySurge.

With built-in optimized data processing, the CData JDBC Driver offers unmatched performance for interacting with live Google Cloud Storage data. When you issue complex SQL queries to Google Cloud Storage, the driver pushes supported SQL operations, like filters and aggregations, directly to Google Cloud Storage and utilizes the embedded SQL engine to process unsupported operations client-side (often SQL functions and JOIN operations). Its built-in dynamic metadata querying allows you to work with and analyze Google Cloud Storage data using native data types.

Connecting to Google Cloud Storage Data in QuerySurge

To connect to live Google Cloud Storage data from QuerySurge, you need to deploy the JDBC Driver JAR file to your QuerySurge Agent(s) and add a new connection from the QuerySurge Admin view.

Deploy the JDBC Driver

1. Download the CData JDBC Driver for Google Cloud Storage installer, unzip the package, and run the JAR file to install the driver.
2. Once the driver is installed, stop the Agent Service.
3. Copy the JAR File (and license file if it exists) from the installation location (typically C:\Program Files\CData\CData JDBC Driver for Google Cloud Storage\lib\) to your Agent(s) (QuerySurge_install_dir\agent\jdbc).
4. Restart the Agent Service.

For more information on deploying JDBC drivers for QuerySurge, refer to the QuerySurge Knowledge Base.

Configure a New Connection to Google Cloud Storage

1. Log into QuerySurge and navigate to the Admin view.
2. Click Configuration -> Connections in the Administration Tree.
3. Click Add to create a new connection.
4. In the QuerySurge Connection Wizard, click Next.
5. Name the connection (e.g. CData JDBC Connection to Google Cloud Storage).
6. Set the Data Source to "All Other JDBC Connections (Connection Extensibility)" and click Next.
7. Set the Driver Class to cdata.jdbc.googlecloudstorage.GoogleCloudStorageDriver and click Next.

8. Set the Connection URL using the necessary connection properties to authenticate with Google Cloud Storage. Your Connection URL will look something like the following:

   jdbc:googlecloudstorage:ProjectId='project1';InitiateOAuth=GETANDREFRESH

   Built-in Connection String Designer

   For assistance in constructing the JDBC URL, use the connection string designer built into the Google Cloud Storage JDBC Driver. Either double-click the JAR file or execute the jar file from the command-line.

   java -jar cdata.jdbc.googlecloudstorage.jar

   Fill in the connection properties and copy the connection string to the clipboard.

   Authenticate with a User Account

   You can connect without setting any connection properties for your user credentials. After setting InitiateOAuth to GETANDREFRESH, you are ready to connect.

   When you connect, the Google Cloud Storage OAuth endpoint opens in your default browser. Log in and grant permissions, then the OAuth process completes

   Authenticate with a Service Account

   Service accounts have silent authentication, without user authentication in the browser. You can also use a service account to delegate enterprise-wide access scopes.

   You need to create an OAuth application in this flow. See the Help documentation for more information. After setting the following connection properties, you are ready to connect:

   • InitiateOAuth: Set this to GETANDREFRESH.
   • OAuthJWTCertType: Set this to "PFXFILE".
   • OAuthJWTCert: Set this to the path to the .p12 file you generated.
   • OAuthJWTCertPassword: Set this to the password of the .p12 file.
   • OAuthJWTCertSubject: Set this to "*" to pick the first certificate in the certificate store.
   • OAuthJWTIssuer: In the service accounts section, click Manage Service Accounts and set this field to the email address displayed in the service account Id field.
   • OAuthJWTSubject: Set this to your enterprise Id if your subject type is set to "enterprise" or your app user Id if your subject type is set to "user".
   • ProjectId: Set this to the Id of the project you want to connect to.

   The OAuth flow for a service account then completes.

9. Set the Test Query to enable the Test Connection button for the Connection (e.g. SELECT * FROM Buckets LIMIT 1) and click Next.
10. Click Test Connection to ensure the connection is configured properly and click Save to add the connection.

Once the connection is added, you can write SQL queries against your Google Cloud Storage data in QuerySurge.

Compare Google Cloud Storage Data Queries with a QueryPair

With the connection configured, you can follow the steps below to compare querying Google Cloud Storage data with a QueryPair.

1. Select "Design Library" from the Design Menu
2. Click QueryPairs under QuerySurge Design
3. Click "Create New QueryPair"
4. Name the QueryPair and click Save
5. In either the Source or Target panes, select the connection created above (select the same connection to query Google Cloud Storage twice or another connection to perform a comparison)
6. Write queries in the Editor for each pane, e.g. SELECT * FROM Buckets
7. Click the "Design-Time Run" tab to execute the queries
8. When the query execution is finished, click "View Query Results" to see the Google Cloud Storage data returned by the query

Download a free, 30-day trial of the CData JDBC Driver for Google Cloud Storage and start working with your live Google Cloud Storage data in QuerySurge. Reach out to our Support Team if you have any questions.