Periodic exports

In the platform, you can setup daily exports to your server.

Configuration#

Go to:

Settings
Account
Periodic exports
+ New

There are two modes:

Specific export: outputs a daily CSV file with either
- all participations for the given campaigns & contents (you can select optins contained in those resources to append their values to each line)
- all selected user optins

After that, pick the BeOp resources you want to export.

Global export: outputs a daily ndJSON file with all the participations and all the optins from all your creatives

Constraints#

For the moment,

Specific exports are only in the CSV format, and are transferred to your distant server using SFTP
Global exports are only in the ndJSON format, and are transferred to your bucket on Google Cloud Storage (GCS). We have a help page if you're unsure about your credentials key and bucket name.

Specific exports#

File naming#

Every day, a new CSV file will arrive in the configured directory on your server.

The file will be named as follows:

YYYYMMDD_TYPE_NAME_UUID.csv

TYPE: crm-optin or crm-participation
NAME: Slugified name (e.g. My export will become my-export)
YYYYMMDD: the current date
UUID: a random ID

As you can chose to run multiple exports every day, we recommend that you pick a different path for each. Don't worry if it's not the case, the random ID will ensure there's no name clash so that all the data is correctly written!

⚠️ As rights strategies can be very different from one case to the other, we will not create the directory, it will need to exist prior to the export.

File contents#

The CSV file will start with the following fixed columns:

email (optional) string
phone (optional) string
firstname (optional) string
lastname (optional) string
gender (optional) string
birthdate (optional) string
address-line-1 (optional) string
address-line-2 (optional) string
address-zip-code (optional) string
address-city (optional) string
address-country (optional) string
creative-id string
creative-name string
campaign-id (optional) string
campaign-name (optional) string

And will end with one column per selected optin, named like the optin:

OPTIN_NAME (e.g. My Newsletter): (optional) true if the user opted in

Global exports#

File naming#

Every day, a new compressed ndJSON file will arrive in the configured directory on your server.

The file will be named as follows:

YYYY-MM-DD_username.jsond.gz

YYYY-MM-DD: the current date
username: your username

File contents#

A newline delimited JSON is a simple text file where each line is a valid json value, here each line corresponds to a participation.

{"questions":[{"question_id" ( ... )
{"questions":[{"question_id" ( ... )
{"questions":[{"question_id" ( ... )
{"questions":[{"question_id" ( ... )
( ... )

Note that if any text value contains a newline character (\n), the latter is escaped (\\n) so that the remainder of the json is not confused with an actual new json entry on a new line. \\n should be handled transparently by any standard json parser.

Each participation json, if one were to expand it on multiple lines for better readability, has the following structure:

{
  "questions" : [
    {
      "question_id" : QUESTION_ID,
      "question_text" : QUESTION_TEXT,
      "anwser_text" : ANSWER_TEXT
    },
    { ... }
  ],
  "email" : EMAIL,
  "optins" : [
    {
      "optin_email": EMAIL,
      "optin_text" : OPTIN_TEXT,
      "optin_id" : OPTIN_ID,
      "optin_value" : true
    },
    { ... }
  ],
  "forms" : [
    {
      "form_id" : FORM_ID,
      "form_name" : FORM_NAME,
      "form_data" : [
        {
          "field_id" : FIELD_ID,
          "field_name" : FIELD_NAME,
          "field_type" : FIELD_TYPE,
          "field_value" : FIELD_VALUE,
        },
        { ... }
      ]
    },
    { ... }
  ],
  "page_url" : URL_PAGE,
  "tc_string" : TC_STRING,
  "timestamp" : TIMESTAMP,
  "publisher_name" : PUBLISHER_NAME
  "publisher_id" : PUBLISHER_ID,
  "content_id" : CONTENT_ID,
  "content_name" : CONTENT_NAME,
  "content_version" : CONTENT_VERSION
}