Automate CSV Integration Tasks from PowerShell

The CData Cmdlets for CSV are standard PowerShell cmdlets that make it easy to accomplish data cleansing, normalization, backup, and other integration tasks by enabling real-time access to CSV.

PowerShell Cmdlets or ADO.NET Provider?

The Cmdlets are not only a PowerShell interface to CSV, but also an SQL interface; this tutorial shows how to use both to retrieve CSV data. We also show examples of the ADO.NET equivalent, which is possible with the CData ADO.NET Provider for CSV. To access CSV data from other .NET applications, like LINQPad, use the CData ADO.NET Provider for CSV.

Once you have acquired the necessary connection properties, accessing CSV data in PowerShell can be enabled in three steps.

The DataSource property must be set to a valid local folder name.

Also, specify the IncludeFiles property to work with text files having extensions that differ from .csv, .tab, or .txt. Specify multiple file extensions in a comma-separated list. You can also set Extended Properties compatible with the Microsoft Jet OLE DB 4.0 driver. Alternatively, you can provide the format of text files in a Schema.ini file.

Set UseRowNumbers to true if you are deleting or updating in CSV. This will create a new column with the name RowNumber which will be used as key for that table.

PowerShell

1. Install the module:

   Install-Module CSVCmdlets

2. Connect:

   $csv = Connect-CSV -DataSource "$DataSource"

3. Search for and retrieve data:

   $firstname = "Bob" $customer = Select-CSV -Connection $csv -Table "Customer" -Where "FirstName = `'$FirstName`'" $customer

   You can also use the Invoke-CSV cmdlet to execute SQL commands:

   $customer = Invoke-CSV -Connection $csv -Query 'SELECT * FROM Customer WHERE FirstName = @FirstName' -Params @{'@FirstName'='Bob'}

ADO.NET

1. Load the provider's assembly:

   [Reflection.Assembly]::LoadFile("C:\Program Files\CData\CData ADO.NET Provider for CSV\lib\System.Data.CData.CSV.dll")

2. Connect to CSV:

   $conn= New-Object System.Data.CData.CSV.CSVConnection("DataSource=MyCSVFilesFolder;") $conn.Open()

3. Instantiate the CSVDataAdapter, execute an SQL query, and output the results:

   $sql="SELECT City, TotalDue from Customer" $da= New-Object System.Data.CData.CSV.CSVDataAdapter($sql, $conn) $dt= New-Object System.Data.DataTable $da.Fill($dt) $dt.Rows | foreach { Write-Host $_.city $_.totaldue }