Automate Google Cloud Storage Integration Tasks from PowerShell

The CData Cmdlets for Google Cloud Storage are standard PowerShell cmdlets that make it easy to accomplish data cleansing, normalization, backup, and other integration tasks by enabling real-time access to Google Cloud Storage.

PowerShell Cmdlets or ADO.NET Provider?

The Cmdlets are not only a PowerShell interface to Google Cloud Storage, but also an SQL interface; this tutorial shows how to use both to retrieve Google Cloud Storage data. We also show examples of the ADO.NET equivalent, which is possible with the CData ADO.NET Provider for Google Cloud Storage. To access Google Cloud Storage data from other .NET applications, like LINQPad, use the CData ADO.NET Provider for Google Cloud Storage.

Once you have acquired the necessary connection properties, accessing Google Cloud Storage data in PowerShell can be enabled in three steps.

Authenticate with a User Account

You can connect without setting any connection properties for your user credentials. After setting InitiateOAuth to GETANDREFRESH, you are ready to connect.

When you connect, the Google Cloud Storage OAuth endpoint opens in your default browser. Log in and grant permissions, then the OAuth process completes

Authenticate with a Service Account

Service accounts have silent authentication, without user authentication in the browser. You can also use a service account to delegate enterprise-wide access scopes.

You need to create an OAuth application in this flow. See the Help documentation for more information. After setting the following connection properties, you are ready to connect:

• InitiateOAuth: Set this to GETANDREFRESH.
• OAuthJWTCertType: Set this to "PFXFILE".
• OAuthJWTCert: Set this to the path to the .p12 file you generated.
• OAuthJWTCertPassword: Set this to the password of the .p12 file.
• OAuthJWTCertSubject: Set this to "*" to pick the first certificate in the certificate store.
• OAuthJWTIssuer: In the service accounts section, click Manage Service Accounts and set this field to the email address displayed in the service account Id field.
• OAuthJWTSubject: Set this to your enterprise Id if your subject type is set to "enterprise" or your app user Id if your subject type is set to "user".
• ProjectId: Set this to the Id of the project you want to connect to.

The OAuth flow for a service account then completes.

PowerShell

1. Install the module:

   Install-Module GoogleCloudStorageCmdlets

2. Connect:

   $googlecloudstorage = Connect-GoogleCloudStorage -ProjectId "$ProjectId"

3. Search for and retrieve data:

   $name = "TestBucket" $buckets = Select-GoogleCloudStorage -Connection $googlecloudstorage -Table "Buckets" -Where "Name = `'$Name`'" $buckets

   You can also use the Invoke-GoogleCloudStorage cmdlet to execute SQL commands:

   $buckets = Invoke-GoogleCloudStorage -Connection $googlecloudstorage -Query 'SELECT * FROM Buckets WHERE Name = @Name' -Params @{'@Name'='TestBucket'}

ADO.NET

1. Load the provider's assembly:

   [Reflection.Assembly]::LoadFile("C:\Program Files\CData\CData ADO.NET Provider for Google Cloud Storage\lib\System.Data.CData.GoogleCloudStorage.dll")

2. Connect to Google Cloud Storage:

   $conn= New-Object System.Data.CData.GoogleCloudStorage.GoogleCloudStorageConnection("ProjectId='project1';InitiateOAuth=GETANDREFRESH") $conn.Open()

3. Instantiate the GoogleCloudStorageDataAdapter, execute an SQL query, and output the results:

   $sql="SELECT Name, OwnerId from Buckets" $da= New-Object System.Data.CData.GoogleCloudStorage.GoogleCloudStorageDataAdapter($sql, $conn) $dt= New-Object System.Data.DataTable $da.Fill($dt) $dt.Rows | foreach { Write-Host $_.name $_.ownerid }