How to work with SAP SuccessFactors Data in Apache Spark using SQL

Apache Spark is a fast and general engine for large-scale data processing. When paired with the CData JDBC Driver for SAP SuccessFactors, Spark can work with live SAP SuccessFactors data. This article describes how to connect to and query SAP SuccessFactors data from a Spark shell.

The CData JDBC Driver offers unmatched performance for interacting with live SAP SuccessFactors data due to optimized data processing built into the driver. When you issue complex SQL queries to SAP SuccessFactors, the driver pushes supported SQL operations, like filters and aggregations, directly to SAP SuccessFactors and utilizes the embedded SQL engine to process unsupported operations (often SQL functions and JOIN operations) client-side. With built-in dynamic metadata querying, you can work with and analyze SAP SuccessFactors data using native data types.

Install the CData JDBC Driver for SAP SuccessFactors

Download the CData JDBC Driver for SAP SuccessFactors installer, unzip the package, and run the JAR file to install the driver.

Start a Spark Shell and Connect to SAP SuccessFactors Data

1. Open a terminal and start the Spark shell with the CData JDBC Driver for SAP SuccessFactors JAR file as the jars parameter: $ spark-shell --jars /CData/CData JDBC Driver for SAP SuccessFactors/lib/cdata.jdbc.sapsuccessfactors.jar
2. With the shell running, you can connect to SAP SuccessFactors with a JDBC URL and use the SQL Context load() function to read a table.

   You can authenticate to SAP Success Factors using Basic authentication or OAuth with SAML assertion.

   Basic Authentication

   You must provide values for the following properties to successfully authenticate to SAP Success Factors. Note that the provider will reuse the session opened by SAP Success Factors using cookies. Which means that your credentials will be used only on the first request to open the session. After that, cookies returned from SAP Success Factors will be used for authentication.

   • Url: set this to the URL of the server hosting Success Factors. Some of the servers are listed in the SAP support documentation (external link).
   • User: set this to the username of your account.
   • Password: set this to the password of your account.
   • CompanyId: set this to the unique identifier of your company.

   OAuth Authentication

   You must provide values for the following properties, which will be used to get the access token.

   • Url: set this to the URL of the server hosting Success Factors. Some of the servers are listed in the SAP support documentation (external link).
   • User: set this to the username of your account.
   • CompanyId: set this to the unique identifier of your company.
   • OAuthClientId: set this to the API Key that was generated in API Center.
   • OAuthClientSecret: the X.509 private key used to sign SAML assertion. The private key can be found in the certificate you downloaded in Registering your OAuth Client Application.
   • InitiateOAuth: set this to GETANDREFRESH.

   Built-in Connection String Designer

   For assistance in constructing the JDBC URL, use the connection string designer built into the SAP SuccessFactors JDBC Driver. Either double-click the JAR file or execute the jar file from the command-line.

   java -jar cdata.jdbc.sapsuccessfactors.jar

   Fill in the connection properties and copy the connection string to the clipboard.

   Configure the connection to SAP SuccessFactors, using the connection string generated above.

   scala> val sapsuccessfactors_df = spark.sqlContext.read.format("jdbc").option("url", "jdbc:sapsuccessfactors:User=username;Password=password;CompanyId=CompanyId;Url=https://api4.successfactors.com;").option("dbtable","ExtAddressInfo").option("driver","cdata.jdbc.sapsuccessfactors.SAPSuccessFactorsDriver").load()
3. Once you connect and the data is loaded you will see the table schema displayed.

4. Register the SAP SuccessFactors data as a temporary table:

   scala> sapsuccessfactors_df.registerTable("extaddressinfo")

5. Perform custom SQL queries against the Data using commands like the one below:

   scala> sapsuccessfactors_df.sqlContext.sql("SELECT address1, zipCode FROM ExtAddressInfo WHERE city = Springfield").collect.foreach(println)

   You will see the results displayed in the console, similar to the following:

Using the CData JDBC Driver for SAP SuccessFactors in Apache Spark, you are able to perform fast and complex analytics on SAP SuccessFactors data, combining the power and utility of Spark with your data. Download a free, 30 day trial of any of the 200+ CData JDBC Drivers and get started today.