製品をチェック

無償トライアル:

無償トライアルへ

製品の情報と無償トライアルへ:

Ponparemall SSIS Components

SSIS ソース元 & 接続先コンポーネントは、SQL Server SSIS のワークフロー内で簡単にPonparemall データにライブ接続できる強力なツールです。

データフロー内のPonparemall コンポーネントを使って在庫、商品、受注、etc. を同期できます。データ同期、ローカルバックアップ、ワークフローの自動化などに最適！

Biml を使って、Ponparemall Data をSQL Server に同期するSSIS タスクを作成

CData SSIS Components を使って、Biml で Ponparemall data をSQL Server に同期するタスクを作成します。

古川えりか
コンテンツスペシャリスト

SSIS

SQL Server に基幹業務データのバックアップを保管することは、ビジネス上のセーフティネットです。また、ユーザーはSQL Server のバックアップデータからレポーティングや分析を簡単に行うことができます。Biml は、SSIS パッケージのようなMicrosoft SQL Server BI Object で使えるXML の方言です。CData SSIS Components をBiml と組み合わせて使うことで、Ponparemall data に連携するSSIS パッケージを簡単ンい作成できます。以下のメリットがあります:

Built-in metadata discovery — CData SSIS components expose metadata just like working with SQL Server, even dynamically generating schema for schema-less data sources
Dynamic SSIS task generation — Use code nuggets in Biml to build SSIS tasks by iterating over discovered metadata
Read from and write to Ponparemall — Native source and destination components make Ponparemall look just like a database

This article demonstrates how to use Biml with the CData SSIS Tasks for Ponparemall to dynamically build SSIS tasks (one for each Ponparemall entity) to replicate Ponparemall data to a Microsoft SQL Server database. We step through the Biml file one section at a time but have included the complete Biml file at the end of the article.

Getting Started

In order to use Biml in an SSIS Project in Visual Studio, install BimlExpress. Once you install BimlExpress, open Visual Studio, create a new Integration Services project, and add a new Biml file.

Building the Biml File

With Biml, you can write scripting to dynamically generate SSIS projects, packages, and tasks. To see the Biml file for an existing project (and gain insights on using Biml with CData SSIS Tasks), simply create your tasks and then right-click the project and select Convert SSIS Packages to Biml.

C# Code

Use directives <#@ .. #> to import necessary namespaces and the assembly for the CData SSIS Components for Ponparemall.

<#@ template language="C#" hostspecific="true"#>
<#@ import namespace="System.Data"#>
<#@ import namespace="System.IO"#>
<#@ import namespace="System.Collections"#>
<#@ import namespace="System.Data.CData.Ponparemall"#>
<#@ assembly name="C:\Program Files\CData\CData SSIS Components for Ponparemall 2018\lib\CData.SSIS2017.Ponparemall.dll"#>

In a new control nugget <# ... #>, create variables for values that will be used throughout the Biml script, including a connection string for Ponparemall and structures to store the Ponparemall metadata.

Ponparemall に接続するには、ShopUrl、InventoryUserId、InventoryPassword、OrderUserId、OrderPassword、PaymentUserId、およびPaymentPassword が必要です。
Ponparemall へのアクセスの設定

Ponparemall には各API のId とPassword が必要です。取得するには、以下の手順に従ってください。
- Ponparemall Manager にログインし、メニューの「各種申請」の「WebAPI利用申請」をクリックします。
- Web API メニューの「WebAPIサービス利用申請」をクリックし、利用申請を行います。
- 後日、Ponparemall から必要なId とPassword が送られてきます。
Ponparemall アカウントの認証

次の接続プロパティを設定して接続します。
- ShopUrl：接続先のPonparemall Shop URL を設定。「http://storetst4.ponparemall.com/sample/」に含まれる「sample」の部分を入力します。
- InventoryUserId：「在庫API」のユーザID を設定。このプロパティは、Inventory テーブルにアクセスする場合に必要です。
- InventoryPassword：「在庫API」のパスワードを設定。このプロパティは、Inventory テーブルにアクセスする場合に必要です。
- OrderUserId：「受注API」のユーザID を設定。このプロパティは、Orders テーブルにアクセスする場合に必要です。
- OrderPassword：「受注API」のパスワードを設定。このプロパティは、Orders テーブルにアクセスする場合に必要です。
- PaymentUserId：「決済API」のユーザID を設定。このプロパティは、Payments テーブルにアクセスする場合に必要です。
- PaymentPassword：「決済API」のパスワードを設定。このプロパティは、Payments テーブルにアクセスする場合に必要です。
```
var ponparemallConnectionString = "InventoryUserId=Inventory UserId;InventoryPassword=Invetory Password;";
var replicationServer = "SERVER";
var replicationCatalog = "CATALOG";
var replicationUserID = "sqluser";
var replicationPassword = "sqlpassword";

List<string> allEntityNames = new List<string>();
Hashtable entitySchema = new Hashtable();
```

In the same control nugget used to defined variables, use ADO.NET code to programmatically query the Ponparemall entities (tables) and fields (columns).

using (PonparemallConnection connection = new PonparemallConnection(ponparemallConnectionString)) {
  connection.Open();
  var entities = connection.GetSchema("Tables").Rows;
  foreach (DataRow entity in entities)
  {
    allEntityNames.Add(entity["TABLE_NAME"].ToString());
  }
  foreach (string entity in allEntityNames){
    var columns = connection.GetSchema("Columns", new string [] {entity}).Rows;
    entitySchema.Add(entity,columns);
  }
}

Class Nugget

In our Biml script to create the replication tasks, there are several places where repeated XML elements are created dynamically (mostly for columns in SSIS tasks). Instead of repeating the code, add a class nugget <#+ ... #> and create a helper class with methods to consolidate repeated code (full code at the end of the article).

Add public static variables to determine which type of XML element to create.

public static int OUTPUT_WITH_ERROR = 0;
public static int EXTERNAL = 1;
public static int OUTPUT = 2;
public static int DATAOVERRIDE_COLUMN = 4;

Add a public method to build a SQL statement for use in the ExecuteSQL task used to drop existing tables and create a new table for the replicated data.

// Dynamically builds a DROP TABLE and CREATE statement
// for each entity (table) in Ponparemall using the table name and metadata.
public static string GetDeleteAndCreateStatement(string tableName, DataRowCollection columns) {
  ...
}

Add a public method to build the collection of column-based XML elements.

 
// Dynamically build various column-based XML elements
// for each entity (table) in Ponparemall based on the column 
// metadata and the parent element
public static string GetColumnDefs(DataRowCollection columns, int columnType){
  ...
}

Biml Script

Now that you have the table metadata and a Helper class to reduce repeated code, write the Biml script to dynamically create your replication packages.

Start by adding a CustomSsisConnection element for the CData SSIS Tasks. Note that the ObjectData attribute must be XML encoded. A typical connecting string looks similar to the following (note the use of the ponparemallConnectionString variable for the ConnectionString property:

<PonparemallConnectionManager>
  <Property Name="ConnectionString"><#=ponparemallConnectionString#></Property>
</PonparemallConnectionManager>

After configuring the connection to the CData SSIS Task, configure a connection to the replication database. The completed Connections element looks like the following (note the use of text nuggets <#= ... #> to add variables for connection string values):

<Connections>
  <CustomSsisConnection Name="CData Ponparemall Connection Manager" CreationName = "CDATA_PONPAREMALL" ObjectData = "&lt;PonparemallConnectionManager&gt; &lt;Property Name=&quot;ConnectionString&quot;&gt; <#=ponparemallConnectionString#>&lt;/Property&gt; &lt;/PonparemallConnectionManager&gt;" />
  <Connection Name="Destination" ConnectionString="Data Source=<#=replicationServer#>;User ID=<#=replicationUserID#>;Password=<#=replicationPassword#>;Initial Catalog=<#=replicationCatalog#>;Provider=SQLNCLI11.1;"/>
</Connections>

With the Connections element configured, you are ready to build our replication package. In the package, the Biml script create an ExecuteSQL task and a Dataflow task for each table to be replicated.

To build each set of tasks, use a while loop in a control nugget to iterate through the entity (table) names:

int entityCounter = 0; while(entityCounter < allEntityNames.Count){
var tableName = allEntityNames[entityCounter].ToString();
DataRowCollection columns = ((DataRowCollection)entitySchema[tableName]);

ExecuteSQL Task
In the ExecuteSQL task, execute a SQL query to drop any existing tables that have the same name as our Ponparemall entity (table) and create a new table based on the metadata discovered using the CData SSIS Component.

To create the query dynamically, use the Helper.GetDeleteAndCreateStatement() helper function.

Dataflow Task

Within the Dataflow use a CustomComponent as the source component and an OleDbDestination as the destination.

CustomComponent Element

The CustomComponent element uses the CData SSIS Source component to retrieve Ponparemall data. Start by configuring the component to use with the CData component.


<CustomComponent Name="CData Ponparemall Source" ComponentTypeName="CData.SSIS.Ponparemall.PonparemallSource" Version="18" ContactInfo="support@cdata.com" UsesDispositions="true">
...
</CustomComponent>

DataflowOverrides and OutputPaths Elements

The next step after configuring the connection is to add Columns elements to the OutputPath child element of the DataflowOverrides element. To do so, call the Helper.GetColumnDefs() helper function.

Use the same Helper class to add columns to the OutputColumns and ExternalColumns child elements of the various OutputPaths elements.

The definitions created provide information about the input, output, and error information for the SSIS component.


<DataflowOverrides>
  <OutputPath OutputPathName="CData Ponparemall Source Output">
    <Columns>
<#=HelperClass.GetColumnDefs(columns,HelperClass.DATAOVERRIDE_COLUMN) #>
    </Columns>
  </OutputPath>
</DataflowOverrides>
...
<OutputPaths>
  <OutputPath Name="CData Ponparemall Source Output">
    <OutputColumns>
<#=HelperClass.GetColumnDefs(columns,HelperClass.OUTPUT_WITH_ERROR) #>
    </OutputColumns>
    <ExternalColumns>
<#=HelperClass.GetColumnDefs(columns,HelperClass.EXTERNAL) #>
    </ExternalColumns>
  </OutputPath>
  <OutputPath Name="CData Ponparemall Source Error Output" IsErrorOutput="true">
    <OutputColumns>
<#=HelperClass.GetColumnDefs(columns,HelperClass.OUTPUT) #      
    </OutputColumns>
  </OutputPath>
</OutputPaths>

CustomProperties Element

The CData SSIS tasks are surfaced in SSIS as custom components with a series of required CustomProperties:


<CustomProperties>
  <CustomProperty Name="SQLStatement" DataType="Null" UITypeEditor="Microsoft.DataTransformationServices.Controls.ModalMultilineStringEditor, Microsoft.DataTransformationServices.Controls, Version= 10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91" SupportsExpression="true"></CustomProperty>
  <CustomProperty Name="AccessMode" DataType="Int32" TypeConverter="CData.SSIS.Ponparemall.AccessModeToStringConverter">0</CustomProperty>
  <CustomProperty Name="TableOrView" DataType="String" UITypeEditor="Microsoft.DataTransformationServices.Controls.ModalMultilineStringEditor, Microsoft.DataTransformationServices.Controls, Version= 10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91" SupportsExpression="true">[<#=tableName#>]</CustomProperty>
  <CustomProperty Name="ExecStoredProcedure" DataType="Boolean">false</CustomProperty>
</CustomProperties>

Connections Element

The last element to add to the CustomComponent element is a Connections element, attaching the previously defined connection to the task:


<Connections>
  <Connection Name="Ponparemall 2018 Connection" ConnectionName="CData Ponparemall Connection Manager" />
</Connections>

OleDbDestination Element

The final piece of the Dataflow task is the OleDbDestination element. Attach the previously defined OleDbConnection to the element, set the InputPath and ExternalTableOutput:


<OleDbDestination Name="OLE DB Destination" ConnectionName="Destination" CheckConstraints="false">
  <InputPath OutputPathName="CData Ponparemall Source.CData Ponparemall Source Output" />
  <ExternalTableOutput Table="[<#=tableName#>]" />
</OleDbDestination>

Use a control nugget to increment the counter used to iterate over the collection of entity (table) names. Do this within the Tasks element, after the end of the Dataflow element:
```
...
          </Dataflow>          
<# entityCounter++;}#>
        </Tasks>
    </Package>
  </Packages>
</Biml>
```

Build the SSIS Project

Once the Biml file is written, right-click on the Biml file in Server Explorer and select Generate SSIS Packages. At this point, Visual Studio and BimlExpress will translate the Biml file into SSIS package(s), ready to be run.

Run the package to begin replicating your Ponparemall data to a SQL Server database (or any other destination you choose).

Free Trial & More Information

With the CData SSIS Components for Ponparemall, you get SQL access to your Ponparemall data directly from SSIS packages. And with Biml, you can automatically generate those packages. For more information about the CData SSIS Components for Ponparemall, refer to the product page. You can always get started with a free, 30-day trial. As always, our world-class CData Support Team is available if you have any questions.

Complete Biml File


<#@ template language="C#" hostspecific="true"#>
<#@ import namespace="System.Data"#>
<#@ import namespace="System.IO"#>
<#@ import namespace="System.Collections"#>
<#@ import namespace="System.Data.CData.Ponparemall"#>
<#@ assembly name="C:\Program Files\CData\CData SSIS Components for Ponparemall 2018\lib\CData.SSIS2017.Ponparemall.dll"#>
<#
var ponparemallConnectionString = ""InventoryUserId=Inventory UserId;InventoryPassword=Invetory Password;";
var replicationServer = "JDG";
var replicationCatalog = "BIML";
var replicationUserID = "sqltest";
var replicationPassword = "sqltest";

List<string> allEntityNames = new List<string>();
Hashtable entitySchema = new Hashtable();
using (PonparemallConnection connection = new PonparemallConnection(ponparemallConnectionString)) {
    connection.Open();
    var entities = connection.GetSchema("Tables").Rows;
    foreach (DataRow entity in entities)
    {
        allEntityNames.Add(entity["TABLE_NAME"].ToString());
    }
    foreach (string entity in allEntityNames){
        var columns = connection.GetSchema("Columns", new string [] {entity}).Rows;
        entitySchema.Add(entity,columns);
    }
}#>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
  <Connections>
    <CustomSsisConnection Name="CData Ponparemall Connection Manager" CreationName="CDATA_PONPAREMALL" ObjectData="&lt;PonparemallConnectionManager&gt;&lt;Property Name=&quot;ConnectionString"&gt;<#=ponparemallConnectionString#>&lt;/Property&gt;&lt;/PonparemallConnectionManager&gt;"/>
    <Connection Name="Destination" ConnectionString="Data Source=<#=replicationServer#>;User ID=<#=replicationUserID#>;Password=<#=replicationPassword#>;Initial Catalog=<#=replicationCatalog#>;Provider=SQLNCLI11.1;"/>
  </Connections>
  <Packages>
    <Package Name="Replicate Ponparemall Package" Language="None" ConstraintMode="LinearOnCompletion" ProtectionLevel="EncryptSensitiveWithUserKey">
      <Tasks>
<# int entityCounter = 0; while(entityCounter < allEntityNames.Count){
   var tableName = allEntityNames[entityCounter].ToString();
   if (tableName.Equals("IdpEventLog")) break;
   DataRowCollection columns = ((DataRowCollection)entitySchema[tableName]);#>
        <ExecuteSQL Name="Create <#=tableName#> Replication Table" ConnectionName="Destination">
          <DirectInput>
<#=HelperClass.GetDeleteAndCreateStatement(tableName,columns)#>
          </DirectInput>
        </ExecuteSQL>
        <Dataflow Name="Replicate <#=tableName#>">
          <Transformations>
            <CustomComponent Name="CData Ponparemall Source" ComponentTypeName="CData.SSIS.Ponparemall.PonparemallSource" Version="18" ContactInfo="support@cdata.com" UsesDispositions="true">
              <DataflowOverrides>
                <OutputPath OutputPathName="CData Ponparemall Source Output">
                  <Columns>
<#=HelperClass.GetColumnDefs(columns,HelperClass.DATAOVERRIDE_COLUMN) #>
                  </Columns>
                </OutputPath>
              </DataflowOverrides>
              <CustomProperties>
                <CustomProperty Name="SQLStatement" DataType="Null" UITypeEditor="Microsoft.DataTransformationServices.Controls.ModalMultilineStringEditor, Microsoft.DataTransformationServices.Controls, Version= 10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91" SupportsExpression="true"></CustomProperty>
                <CustomProperty Name="AccessMode" DataType="Int32" TypeConverter="CData.SSIS.Ponparemall.AccessModeToStringConverter">0</CustomProperty>
                <CustomProperty Name="TableOrView" DataType="String" UITypeEditor="Microsoft.DataTransformationServices.Controls.ModalMultilineStringEditor, Microsoft.DataTransformationServices.Controls, Version= 10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91" SupportsExpression="true">[<#=tableName#>]</CustomProperty>
                <CustomProperty Name="ExecStoredProcedure" DataType="Boolean">false</CustomProperty>
              </CustomProperties>
              <OutputPaths>
                <OutputPath Name="CData Ponparemall Source Output">
                  <OutputColumns>
<#=HelperClass.GetColumnDefs(columns,HelperClass.OUTPUT_WITH_ERROR) #>
                  </OutputColumns>
                  <ExternalColumns>
<#=HelperClass.GetColumnDefs(columns,HelperClass.EXTERNAL) #>
                  </ExternalColumns>
                </OutputPath>
                <OutputPath Name="CData Ponparemall Source Error Output" IsErrorOutput="true">
                  <OutputColumns>
<#=HelperClass.GetColumnDefs(columns,HelperClass.OUTPUT) #>                     
                  </OutputColumns>
                </OutputPath>
              </OutputPaths>
              <Connections>
                <Connection Name="Ponparemall 2018 Connection" ConnectionName="CData Ponparemall Connection Manager" />
              </Connections>
            </CustomComponent>
            <OleDbDestination Name="OLE DB Destination" ConnectionName="Destination" CheckConstraints="false">
              <InputPath OutputPathName="CData Ponparemall Source.CData Ponparemall Source Output" />
              <ExternalTableOutput Table="[<#=tableName#>]" />
            </OleDbDestination>
          </Transformations>
        </Dataflow>          
<# entityCounter++;}#>
      </Tasks>
    </Package>
  </Packages>
</Biml>

<#+
public static class HelperClass {
    
    public static int OUTPUT_WITH_ERROR = 0;
    public static int EXTERNAL = 1;
    public static int OUTPUT = 2;
    public static int DATAOVERRIDE_COLUMN = 4;
    
    public static string GetDeleteAndCreateStatement(string tableName, DataRowCollection columns) {
        var dropAndCreateStatement = 
            "IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[{0}]') AND type IN (N'U'))\r\n" + 
            "DROP TABLE [{0}];\r\n" + 
            "CREATE TABLE [{0}]\r\n" + 
            "(\r\n" + 
            "{1}\r\n" + 
            ")\r\n" + 
            "ON \"default\";";
        string columnDefs = "";
        foreach (DataRow column in columns){
            string columnDef = "    [{0}] {1}";
            string dataType = column["DATA_TYPE"].ToString();
            if (dataType.ToLower().StartsWith("bool")) {
                dataType = "bit";
            } else if (dataType.ToLower().Equals("real")) {
                dataType = "float";
            } else if (dataType.ToLower().Contains("varchar")) {
                var columnLength = column["CHARACTER_MAXIMUM_LENGTH"];
                dataType = "nvarchar(" + ((int)columnLength > 4000 ? "MAX" : columnLength) + ")";         
            } 
            columnDefs += String.Format(columnDef,column["COLUMN_NAME"],dataType) + ",\r\n";
            
        }
        columnDefs = columnDefs.Remove(columnDefs.LastIndexOf(",\r\n"),",\r\n".Length);
        return String.Format(dropAndCreateStatement,tableName,columnDefs);
    }
    
    public static string GetColumnDefs(DataRowCollection columns, int columnType){
        var columnDefTemplate = "";
        var columnElements = "";
        
        if (columnType == DATAOVERRIDE_COLUMN) {
            columnDefTemplate = "                      <Column ErrorRowDisposition=\"FailComponent\" TruncationRowDisposition=\"FailComponent\" ColumnName=\"{0}\" />\r\n";
            foreach(DataRow column in columns) {
                var columnName = column["COLUMN_NAME"];
                columnElements += String.Format(columnDefTemplate,columnName);
            }
            return columnElements;
        } 
        if (columnType == OUTPUT_WITH_ERROR)
            columnDefTemplate = "                      <OutputColumn Name=\"{0}\" {1} ExternalMetadataColumnName=\"{0}\" ErrorRowDisposition=\"FailComponent\" TruncationRowDisposition=\"FailComponent\" />\r\n";
        else if (columnType == EXTERNAL)
            columnDefTemplate = "                      <ExternalColumn Name=\"{0}\" {1} />\r\n";
        else if (columnType == OUTPUT)
            columnDefTemplate = "                      <OutputColumn Name=\"{0}\" {1} />\r\n";
        
        foreach(DataRow column in columns){ 
            var columnName = column["COLUMN_NAME"];
            var dataTypeRaw = column["DATA_TYPE"].ToString().ToLower();
            var typeAndRelatedInfo = "";
            if (dataTypeRaw.Equals("bool")) {
                typeAndRelatedInfo = "DataType=\"Boolean\"";
            } else if (dataTypeRaw.Equals("date")) {
                typeAndRelatedInfo = "DataType=\"Date\" SsisDataTypeOverride=\"DT_DBDATE\"";
            } else if (dataTypeRaw.Equals("datetime")) {
                typeAndRelatedInfo = "DataType=\"DateTime\"";
            } else if (dataTypeRaw.Equals("real")) {
                typeAndRelatedInfo = ((int)column["NumericPrecision"] > 0 ? "Precision=\"18\" " : " ") + ((int)column["NumericScale"] > 0 ? "Scale=\"15\" " : " ") + "DataType=\"Decimal\"";
            } else if (dataTypeRaw.Equals("varchar")) {
                var columnLength = column["CHARACTER_MAXIMUM_LENGTH"];
                if ((int)columnLength > 4000) {
                    typeAndRelatedInfo = "DataType=\"String\"";
                } else {
                    typeAndRelatedInfo = "Length=\"" + columnLength + "\" DataType=\"String\" CodePage=\"1252\"";
                }
            }
            columnElements += String.Format(columnDefTemplate,columnName,typeAndRelatedInfo);
        }
        return columnElements;
    }
}
#>

CData Software は、データアクセスおよびデータ接続ソリューションのリーディングプロバイダーです。CData の標準コネクタはあらゆるツール・ミドルウェアからのSaaS やDB データの連携を簡単にします。

お問い合わせ

製品概要

データコネクタ

ETL/ ELT ソリューション

クラウド&API 接続

OEM & カスタムドライバー開発

お問い合わせ先

製品を知る

データ分析 & BI

企業情報

ニュース & 事例

製品

ソリューション

コネクタ

サポート

価格

企業情報

リソース

製品

テクノロジー

FEATURED

用途から探す

ソリューション

開催予定のイベント

データソースカテゴリ

人気のコネクタ

ビッグデータ & NoSQL コネクタ

ERP & CRM コネクタ

会計コネクタ

マーケティング & アナリティクスコネクタ

コラボレーションコネクタ

ファイル & API 連携コネクタ

EC コネクタ

リレーショナルデータベースコネクタ

サポート

最新の記事

価格

企業情報

導入事例

リソース

ブログへ →

最新記事

ナレッジベース

製品をチェック

記事の内容

関連記事

Biml を使って、Ponparemall Data をSQL Server に同期するSSIS タスクを作成

CData SSIS Components を使って、Biml で Ponparemall data をSQL Server に同期するタスクを作成します。

Getting Started

Building the Biml File

C# Code

Ponparemall へのアクセスの設定

Ponparemall アカウントの認証

Class Nugget

Biml Script

DataflowOverrides and OutputPaths Elements

CustomProperties Element

Connections Element

Build the SSIS Project

Free Trial & More Information

Complete Biml File

お問い合わせ

お問い合わせ先