Powershell からの Azure Data Lake Storage データ連携タスクを自動化

詳細情報をご希望ですか?

無償トライアル:

ダウンロードへ

製品の詳細情報へ:

Azure Data Lake Storage ADO.NET Provider

Azure Data Lake Storage データに連携する.NET アプリケーションを素早く、簡単に開発できる便利なドライバー。



PowerShell から Azure Data Lake Storage に簡単に接続する方法をお探しですか? CData ADO.NET Provider for Azure Data Lake Storage は、PowerShell スクリプトの優位性を使い、シンプルで簡単に使えるADO.NET インターフェース を提供します。PowerShell スクリプトで、ADO.NET オブジェクトを使って簡単にAzure Data Lake Storage に接続して、同期、自動化、ダウンロードなどが可能!



CData ADO.NET Provider for Azure Data Lake Storage は、ADO.NET 標準インターフェースへ統合し、PowerShell のような. NET アプリケーションからAzure Data Lake Storage API へのデータ連携を可能にします。このプロバイダーは、Azure Data Lake Storage の認証および相互作用を簡単にします。このチュートリアルでは、PowerShell から直接SQL クエリを実行するための、いくつかの一般的なADO.NET オブジェクトの使い方を説明します。

CRUD コマンドの実行

次の3つのステップに従って SELECT クエリ PowerShell からリアルタイムAzure Data Lake Storage に実行:

  1. プロバイダーのアセンブリをロード:

    [Reflection.Assembly]::LoadFile("C:\Program Files\CData\CData ADO.NET Provider for Azure Data Lake Storage\lib\System.Data.CData.ADLS.dll")
  2. Azure Data Lake Storage に接続:

    Authenticating to a Gen 1 DataLakeStore Account

    Gen 1 uses OAuth 2.0 in Azure AD for authentication.

    For this, an Active Directory web application is required. You can create one as follows:

    1. Sign in to your Azure Account through the .
    2. Select "Azure Active Directory".
    3. Select "App registrations".
    4. Select "New application registration".
    5. Provide a name and URL for the application. Select Web app for the type of application you want to create.
    6. Select "Required permissions" and change the required permissions for this app. At a minimum, "Azure Data Lake" and "Windows Azure Service Management API" are required.
    7. Select "Key" and generate a new key. Add a description, a duration, and take note of the generated key. You won't be able to see it again.

    To authenticate against a Gen 1 DataLakeStore account, the following properties are required:

    • Schema: Set this to ADLSGen1.
    • Account: Set this to the name of the account.
    • OAuthClientId: Set this to the application Id of the app you created.
    • OAuthClientSecret: Set this to the key generated for the app you created.
    • TenantId: Set this to the tenant Id. See the property for more information on how to acquire this.
    • Directory: Set this to the path which will be used to store the replicated file. If not specified, the root directory will be used.

    Authenticating to a Gen 2 DataLakeStore Account

    To authenticate against a Gen 2 DataLakeStore account, the following properties are required:

    • Schema: Set this to ADLSGen2.
    • Account: Set this to the name of the account.
    • FileSystem: Set this to the file system which will be used for this account.
    • AccessKey: Set this to the access key which will be used to authenticate the calls to the API. See the property for more information on how to acquire this.
    • Directory: Set this to the path which will be used to store the replicated file. If not specified, the root directory will be used.

    $constr = "Schema=ADLSGen2;Account=myAccount;FileSystem=myFileSystem;AccessKey=myAccessKey;" $conn= New-Object System.Data.CData.ADLS.ADLSConnection($constr) $conn.Open()
  3. ADLSDataAdapter のインスタンスを生成してSQL クエリを実行し、結果を出力:

    $sql="SELECT FullPath, Permission from Resources" $da= New-Object System.Data.CData.ADLS.ADLSDataAdapter($sql, $conn) $dt= New-Object System.Data.DataTable $da.Fill($dt) $dt.Rows | foreach { Write-Host $_.fullpath $_.permission }