Data Exporter - Overview

The Data Exporter is a tool designed to provide a set of flat files for customers to consume the data from the various reporting views in the Real-time Data Warehouse (RTDW). Data Exporter does not currently export all data from Real-time Data Warehouse. Data Exporter dynamically pulls customer-related data across the various Cornerstone modules and delivers them to the Cornerstone FTP for the customer to consume. This process can completed daily, weekly, or monthly, but the recommended frequency is daily.

Data Exporter can be enabled in all environments (i.e., Production, Stage, Pilot).

Data Exporter creates specially crafted .zip files, and data can be exported in the following formats:

• CSV (RFC 4180-compliant)
• JSON/NDJSON

To enable your data to be exported in JSON format, please contact Global Customer Support.

Data Exporter compresses the data, and it can also encrypt the data using AES 256. AES is a symmetric-key algorithm, meaning the same key is used for both encrypting and decrypting the data.

Implementation

To implement Data Exporter, create a Service Request to Global Customer Support to enable the Data Exporter. Global Customer Support will respond with a questionnaire to be completed. It requests the following information:

1. Environments - Specify the environments in which Data Exporter should be enabled.
2. Schedule - Specify the timing of data export.
3. Password - This is an optional password for opening the .zip files. Do not put the password in the questionnaire. Please place the password within a text file that is on the Cornerstone FTP. Provide the path to that file in the questionnaire.
4. Special FTP Directory - This is optional. Specify a special directory on the FTP for exported data files.
5. Data Exporter Objects - Select the objects that should be exported.

Provide the completed questionnaire to Global Customer Support. Global Customer Support will enable the Data Exporter.

API Integration Guide

Click here to download the Data Exporter API Integration Guide.

Deliverables

When a customer uses Data Exporter, they receive .zip files that will work with most archival apps and PKZIP compliant APIs. One .zip file is included for each Data Exporter object. The files are placed in the report delivery folder (Reports) under the following subfolder: DataExports/{Alias}/v{version#}

The Data Exporter creates a new folder for each extract. The folder name is an internal version number that is incremental but not necessarily sequential.

The system tracks the changes since the previous export and does not produce an output if the current version of the object is identical to a previous version. This reduces load by avoiding unnecessary data movement and processing. However, this also means that the consumer should not truncate existing data until the existence of a newer output version is observed.

The export process writes a 0-byte file with the name "{portal name}_v{version}_COMPLETE" to denote the completion of an export. If the folder does not contain this file, it should not be considered a valid data set.

ZIP File

The .zip file that is delivered by Data Exporter includes the following items:

• Data - This may be in CSV, JSON, or NDJSON format.
• Manifest - This is in JSON format, and it contains the metadata.

To enable your data to be exported in JSON format, please contact Global Customer Support.

Identifying New or Updated Data Records

Data that has changed since the last time it was retrieved, can be extracted for certain Data Exporter objects. There are two possible mechanisms to do this:

• Delta Exports - Data Exporter can provide delta exports for certain objects (i.e., only provide data that has changed since the last export). Note that this feature is not available for all the objects. Please review the latest technical specification available in the Community to get an overview of objects supporting delta exports.
• _Last_Touched_DT_UTC - Certain objects contain a “_last_touched_dt_utc” field. This field represents the date and time in UTC when the record was added or modified in the Real-Time Data Warehouse (RTDW). Customers can use this field to identify data that has been added or modified. Please review the latest technical specification available in the Community to get an overview of objects supporting this field.

Frequently Asked Questions (FAQs)