Google Cloud Storage (GCS)

Setting up your Google Cloud Storage bucket destination.

Creating a GCS Destination

From the UI

You can create a new GCS destination from your DCN by going to the Integrations -> Destinations section in your navigation menu or by using the CLI. (see above tab).

To add your GCS bucket, you will need to enter the name of this destination (this is how your destination will appear in the listing), the GCS bucket URL and your GCS credentials file.

The credentials file or<credentials-path> via the CLI is required and should refer to a GCP service account's credential JSON file, which you can download from IAM. The service account must have at least write permissions to the specified bucket (storage.objects.create).

Adding Your GCS Destination

From the CLI

From the CLI, enter the following:

optable-cli destination create gcs <bucket> <credentials-path>

Where

  <bucket>              Bucket URL in which objects should be stored.
  <credentials-path>    Path to Google credentials file.

For example:

$ optable-cli destination create gcs my-gs-bucket-name \
              /path/to/credentials.json

Optional Arguments

By default, created destination have the same name as the specified bucket name. Because destination names must be unique in the DCN, if you wish to create multiple destinations with the same bucket name, you can specify a custom name for the destination itself with the --name=<name> option.

Here is an example of the expected CLI output:

{
  "id": 7,
  "kind": "DESTINATION_KIND_GCS",
  "created_at": "2022-09-28T14:27:46.412692412Z",
  "updated_at": "2022-09-28T14:27:46.412692412Z",
  "name": "destinationfromCLI",
  "state": "DESTINATION_STATE_ACTIVE",
  "gcs": {
    "bucket": "gs://destinationfromCLI"
  }
}

Finally, if you created the destination using your DCN UI you will be redirected to the destinations listing. If you used the CLI, you will be able to export directly from the at the command line.

Exporting Audiences to GCS Buckets.

From the UI

To export to your GCS bucket, navigate to the audience you want to export and click "Export in the top right. From there you will be asked to select your preferred GCS export destination from a list of all available destinations.

Here you can select what format to export your audience as, both CSV or TSV formats are supported. In addition to the format, you can choose to export either IDs or ID clusters, as well as which IDs to export and various export options. At the bottom of the export page, you can set the name of the file that will be exported to your GCS bucket.

From the CLI

From the CLI, enter the following:

optable-cli export create file <audience-id> <destination-id> <filename> <format>

Where:

  <audience-id>       Audience to export.
  <destination-id>    Destination of the export.
  <filename>          Filename of the Export on the destination.
  <format>            Format to use for export (id_csv, id_tsv, cluster_json).

You can choose which ID to send with:

--identifiers-filter=IDENTIFIERS-FILTER,...
                             Filter identifiers by type. Options are: idfa,
                             gaid. Defaults to all types if omitted.

Here is an example of what your command should look like:

$ optable-cli export create file 114 7 "myfile.csv" "id_csv"

Here is an example of the expected CLI output:

{
  "id": 95,
  "created_at": "2022-09-28T14:40:22.878863747Z",
  "updated_at": "2022-09-28T14:40:22.878863747Z",
  "destination": {
    "id": 7,
    "kind": "DESTINATION_KIND_GCS",
    "created_at": "2022-09-28T14:27:46.412692Z",
    "updated_at": "2022-09-28T14:27:46.412692Z",
    "name": "destinationfromCLI",
    "state": "DESTINATION_STATE_ACTIVE"
  },
  "audience": {
    "id": 114,
    "name": "myAudience",
    "kind": "AUDIENCE_KIND_QUERY",
    "created_at": "2022-07-15T17:53:29.255148Z",
    "updated_at": "2022-07-15T17:56:27.565118Z",
    "insights_computed_at": "2022-09-07T14:03:06.979720Z",
    "state": "AUDIENCE_STATE_ACTIVE",
    "activation": {
      "keyword": "48dcdse8ff"
    }
  },
  "state": "EXPORT_STATE_PENDING",
  "identifiers_filter": [
    "ID_KIND_EMAIL_HASH",
    "ID_KIND_APPLE_IDFA",
    "ID_KIND_GOOGLE_GAID",
    "ID_KIND_CUSTOM_ID",
    "ID_KIND_VISITOR_ID",
  ],
  "file": {
    "filename": "myfile.csv",
    "override_existing": true,
    "export_metadata": true,
    "id_format": {
      "kind": "ID_FORMAT_KIND_CSV",
      "type_column": true,
      "include_header": true
    }
  },
  "filename": "myfile.csv",
  "override_existing": true,
  "export_metadata": true,
  "id_format": {
    "kind": "ID_FORMAT_KIND_CSV",
    "type_column": true,
    "include_header": true
  }
}

What Can I Export to GCS?

You can export IDs only or ID clusters. All ID types are supported by this destination type. Exports to this destination also have the following optional settings:

• Include the header row.

• Include values prefixed with type indicator.

• ID type in second column.

• Export an additional METADATA file

How Will my Export Appear in GCS?

Your export will appear in your GCS bucket with the file name you set during export creation.

If you choose to re-export an audience with the same name, it will overwrite the existing file in your storage bucket. However, if you opt for a filename that does not currently exist, a new file will be created upon export.