Identifier Profiles

Input format describing an identifier and its associated data

What are Identifier Profiles?

An identifier ("ID") profile is a JSON input data format describing an ID and its associated data. An ID may be linked with one or more neighboring IDs. An ID may also have traits associated with it. An ID profile can be used to describe an identity mapping between two or more identifiers, and/or to load various traits associated with an ID.

You can load ID profile records into your DCN via the source API, the Optable SDKs, by uploading a file or through our various app integrations. More details can be found in the sources section of the documentation.

Your Optable DCN automatically assembles identity clusters from loaded ID profiles.

Loading ID Profiles

An ID profile is a JSON object containing the following attributes:

• The "id" attribute value is equal to a type-prefixed ID string.

• Optionally, the "neighbors" attribute value is an array of objects, each containing a single "id" attribute value equal to a type-prefixed ID string. Each neighboring ID is considered to be linked with the ID specified by the top-level "id" attribute.

• Optionally, the "traits" attribute value is an array of objects, each containing "key" and "value" attributes. For each trait:

  • The "key" attribute is a string equal to the name of the trait.

  • The "value" attribute is a string equal to the value of the trait. The string type can be modified after ingestion.

Warning: ID Profile Attribute Size Limits

The maximum number of supported neighbors in a single ID profile JSON record is 64.

The maximum number of supported traits in a single ID profile JSON is 64.

Individual traitkey strings must be at least 1 character and at most 128 characters long.

Individual traitvalue strings must be at least 1 character and at most 8192 characters long.

The id attribute value, both at the top-level of the ID profile as well as within the neighbors list, must reference a valid identifier type.

ID profile JSON records which contain more than the limit number of neighbors or traits will error on load.

Example ID Profile Input JSON

Here is an example of a single ID profile, in pretty-printed JSON:

{
  "id": "e:3b2b95279a401a9e27125c588fbe72cda4a2fe9bb551684ea3a11d63be0a63e1",
  "neighbors": [
    {
      "id": "c1:ed1b94a0c6ea1095a439d8399734e47379c204c4fbcf8aeaf6f8594da4e72873"
    },
    {
      "id": "c:0ujsswThIGTUYm2K8FjOOfXtY1K"
    }
  ],
  "traits": [
    {
      "key": "foo",
      "value": "1234"
    },
    {
      "key": "bar",
      "value": "baz"
    }
  ]
}

The ID profile above is related to the hashed Email ID e:3b2b95279a401a9e27125c588fbe72cda4a2fe9bb551684ea3a11d63be0a63e1 which is linked with two custom IDs, c:0ujsswThIGTUYm2K8FjOOfXtY1K and c1:ed1b94a0c6ea1095a439d8399734e47379c204c4fbcf8aeaf6f8594da4e72873

There are also a couple of key value traits associated with the ID profile, namely "foo": "1234" and "bar": "baz"

Loading ID Profiles From API Sources

You can load ID profiles via an API source. Once you've created the API source you can stream in a batch of ID profiles using the CLI. Create a file with one JSON ID profile per line, and run optable-cli source push <source-id> --file=/file/path/to/stream/from.json You can also just stream them in from stdin by omitting --file=...

The CLI source push command batches your records into groups of 1,024. If an error is encountered in a record, the importing of the batch in which that record is will be rejected, but subsequent batches will be loaded. Because loading of ID profiles is effectively an update or insert (upsert) operation, you can simply reload your ID profile record file once you've corrected the erroneous record. You can also provide the --fail-fast argument to source push if you want the CLI to stop loading when it encounters an error.

You can also load ID profiles without neighbors or traits from a newline delimited list of type-prefixed identifiers by running the CLI api-source push command with --format="id", for example: optable-cli api-source push <source-id> --format="id" <<<"c:123\nc:456". The same ID list uploaded into a file upload could be streamed in via an api-source in this manner.

Loading ID Profiles From SDK Sources

If you've deployed the SDK on your site or app, visitor ID profiles are automatically loaded into your DCN in real-time, as a consequence of calling the SDK identify API. Optionally neighboring IDs can be sent as arguments.

You can also associate traits with a visitor's ID by calling the SDK profile API.

Refer to the SDK READMEs on GitHub for additional information on the identify and profile APIs, as well as links to demos and usage examples.

Loading ID Profiles by File Upload

When you upload a file into a source, an ID profile is loaded for each ID appearing in the file.

Incremental Upserts

Note that ID profiles loads are treated by your DCN as upserts, meaning that if the ID profile does not already exist in the DCN, it will be created with the specified attributes. If the ID profile already exists, it will be updated to include the specified attributes in addition to any other existing attributes that have previously been loaded. Each neighbor and trait association is tagged with the information identifying the source which has created it as explained in the Source Tags section below.

Thanks to the upsert behavior, you can load IDs and their associated data into your DCN incrementally, in parallel, and from multiple sources.

Note that several sources can contribute values for the same trait, as well as contribute the same ID links (neighbors). The data contributed by each source is recorded as part of the ID profile separately. When identity clusters are computed from ID profiles by your DCN, any conflicting trait key values are resolved.

Deleting ID Profiles

When you delete a source, all ID profiles that were previously loaded by that source are deleted within 24 hours. Similarly, when an ID profile hasn't been updated or touched for longer than a source's configured expiry period, it will be automatically deleted.

The deletion of an ID profile is scoped to a source and data from each source is recorded separately.

Note that deleting an ID profile loaded from one source may result in an ID profile describing the same identifier, and which was previously loaded through another source, to become visible in identity cluster data. Consider the case where the same ID profile is loaded from two sources but with different trait values. If the second source loads after the first, then the trait value specified by the second source will take precedence due to trait key conflict resolution. If the second source is deleted, the trait value specified by the first source will be selected during identity cluster resolution.