Identifier Profiles

Input format describing an identifier and its associated data

What are Identifier Profiles?

An identifier ("ID") profile is a JSON input data format describing an ID and its associated data. An ID may be linked with one or more neighboring IDs. An ID may also have traits associated with it. An ID profile can be used to describe an identity mapping between two or more identifiers, and/or to load various traits associated with an ID.

You can load ID profile records into your DCN via the source API, the Optable SDKs, by uploading a file or through our various app integrations. More details can be found in the sources section of the documentation.

Your Optable DCN automatically assembles identity clusters from loaded ID profiles.

Loading ID Profiles

An ID profile is a JSON object containing the following attributes:

The "id" attribute value is equal to a type-prefixed ID string.
Optionally, the "scope" attribute value is equal to either "person" or "household", and used to indicate the identity resolution scope of the ID. If not specified, the default "person" scope is assumed.
Optionally, the "neighbors" attribute value is an array of objects, each containing a single "id" attribute value equal to a type-prefixed ID string. Each neighboring ID is considered to be linked with the ID specified by the top-level "id" attribute.
Optionally, the "traits" attribute value is an array of objects, each containing "key" and "value" attributes. For each trait:
- The "key" attribute is a string equal to the name of the trait.
- The "value" attribute is a string equal to the value of the trait. The string type can be modified after ingestion.

Warning: ID Profile Attribute Size Limits

The maximum number of supported neighbors in a single ID profile JSON record is 64.

The maximum number of supported traits in a single ID profile JSON is 64.

Individual traitkey strings must be at least 1 character and at most 128 characters long.

Individual traitvalue strings must be at least 1 character and at most 8192 characters long.

The id attribute value, both at the top-level of the ID profile as well as within the neighbors list, must reference a valid identifier type.

ID profile JSON records which contain more than the limit number of neighbors or traits will error on load.

Example ID Profile Input JSON

Here is an example of a single ID profile, in pretty-printed JSON:

{
  "id": "e:3b2b95279a401a9e27125c588fbe72cda4a2fe9bb551684ea3a11d63be0a63e1",
  "neighbors": [
    {
      "id": "c1:ed1b94a0c6ea1095a439d8399734e47379c204c4fbcf8aeaf6f8594da4e72873"
    },
    {
      "id": "c:0ujsswThIGTUYm2K8FjOOfXtY1K"
    }
  ],
  "traits": [
    {
      "key": "foo",
      "value": "1234"
    },
    {
      "key": "bar",
      "value": "baz"
    }
  ]
}

The ID profile above is related to the hashed Email ID e:3b2b95279a401a9e27125c588fbe72cda4a2fe9bb551684ea3a11d63be0a63e1 which is linked with two custom IDs, c:0ujsswThIGTUYm2K8FjOOfXtY1K and c1:ed1b94a0c6ea1095a439d8399734e47379c204c4fbcf8aeaf6f8594da4e72873

There are also a couple of key value traits associated with the ID profile, namely "foo": "1234" and "bar": "baz"

Loading ID Profiles From API Sources

You can load ID profiles via an API source. Once you've created the API source you can stream in a batch of ID profiles using the CLI. Create a file with one JSON ID profile per line, and run optable-cli source push <source-id> --file=/file/path/to/stream/from.json You can also just stream them in from stdin by omitting --file=...

The CLI source push command batches your records into groups of 1,024. If an error is encountered in a record, the importing of the batch in which that record is will be rejected, but subsequent batches will be loaded. Because loading of ID profiles is effectively an update or insert (upsert) operation, you can simply reload your ID profile record file once you've corrected the erroneous record. You can also provide the --fail-fast argument to source push if you want the CLI to stop loading when it encounters an error.

You can also load ID profiles without neighbors or traits from a newline delimited list of type-prefixed identifiers by running the CLI api-source push command with --format="id", for example: optable-cli api-source push <source-id> --format="id" <<<"c:123\nc:456". The same ID list uploaded into a file upload could be streamed in via an api-source in this manner.

Loading ID Profiles From SDK Sources

If you've deployed the SDK on your site or app, visitor ID profiles are automatically loaded into your DCN in real-time, as a consequence of calling the SDK identify API. Optionally neighboring IDs can be sent as arguments.

You can also associate traits with a visitor's ID by calling the SDK profile API.

Refer to the SDK READMEs on GitHub for additional information on the identify and profile APIs, as well as links to demos and usage examples.

Loading ID Profiles by File Upload

When you upload a file into a source, an ID profile is loaded for each ID appearing in the file.

Incremental Upserts

Note that ID profiles loads are treated by your DCN as upserts, meaning that if the ID profile does not already exist in the DCN, it will be created with the specified attributes. If the ID profile already exists, it will be updated to include the specified attributes in addition to any other existing attributes that have previously been loaded. Each neighbor and trait association is tagged with the information identifying the source which has created it as explained in the Source Tags section below.

Thanks to the upsert behavior, you can load IDs and their associated data into your DCN incrementally, in parallel, and from multiple sources.

Note that several sources can contribute values for the same trait, as well as contribute the same ID links (neighbors). The data contributed by each source is recorded as part of the ID profile separately. When identity clusters are computed from ID profiles by your DCN, any conflicting trait key values are resolved.

Deleting ID Profiles

When you delete a source, all ID profiles that were previously loaded by that source are deleted within 24 hours. Similarly, when an ID profile hasn't been updated or touched for longer than a source's configured expiry period, it will be automatically deleted.

The deletion of an ID profile is scoped to a source and data from each source is recorded separately.

Note that deleting an ID profile loaded from one source may result in an ID profile describing the same identifier, and which was previously loaded through another source, to become visible in identity cluster data. Consider the case where the same ID profile is loaded from two sources but with different trait values. If the second source loads after the first, then the trait value specified by the second source will take precedence due to trait key conflict resolution. If the second source is deleted, the trait value specified by the first source will be selected during identity cluster resolution.

Identity Resolution Scopes

By default, an ID profile describes identifiers, traits, and links to neighboring identifiers that will be resolved to a single identity cluster representing an individual person. In other words, linking two or more identifiers together into a single identity cluster implies that they all belong to that same person.

Household Data

When an identifier refers to a household device like a home wifi router or streaming TV device shared by multiple people, it is more appropriate to associate it with not one but multiple person identity clusters. In order to do this, you can specify the "scope" attribute in the ID profile JSON and set it equal to the "household" value.

{
  "id": "c2:a9e27125c588fbe1",
  ...
  "scope": "household",
  ...
}

The household scope only applies to identifiers. Traits loaded alongside household scoped IDs is still considered person scope. Please refer to the Identity Resolution section for more details.

The permitted values for "scope" are "household" or "person". If the scope is not explicitly specified in the ID profile JSON, the default "person" scope is assumed.

It's not currently possible to load household data using the SDKs.

You can load household-scoped ID profiles from an API source, File Upload or by using the Optable CLI.

Neighboring Identifiers

When an ID profile refers to neighboring IDs via the "neighbors" attribute, any of the neighboring IDs that are seen for the first time by your DCN will be automatically created with the default person scope, even if the "scope" explicitly specified in the ID profile is "household".

When an ID profile refers to neighboring IDs that have already been loaded, the scope of those IDs is not modified.

Changing Scope

You can change the scope of an already loaded ID profile by loading another ID profile referencing the same ID and explicitly specifying the new desired scope. The last successfully loaded ID profile's "scope" value is retained replacing any previously loaded value, if any.

If you want to change the scope of an ID profile to a person level scope, must specify "scope" explicitly and set it to "person". If you omit the "scope" attribute from the ID profile, the default will not be to overwrite the previously assigned scope, so you must specify it explicitly.

Example

For example, given the following two ID profiles:

{
  "id": "e:3b2b95279a401a9e27125c588fbe72cda4a2fe9bb551684ea3a11d63be0a63e1",
  ...
  "scope": "person",
  "neighbors": [ "c2:a9e27125c588fbe1" ]
  ...
}

and:

{
  "id": "c2:a9e27125c588fbe1",
  ...
  "scope": "household",
  ...
}

If the first person level identifier's profile is loaded first, it will be created with "person" scope, and its associated and newly created neighbor ID "c2:a9e27125c588fbe1" will inherit the same (person) scope. Once the second identifier profile referencing "c2:a9e27125c588fbe1" is loaded it will override the previously inherited scope to "household".

If the second ID profile referencing "c2:a9e27125c588fbe1" is loaded first, it will be created with the specified "household" scope. When the first ID profile is subsequently loaded, it will be created with the "person" scope as specified, but the referenced neighbor ID "c2:a9e27125c588fbe1" will be left with household scope since it already exists (as explained in the Neighboring Identifiers section above).

Effect on Identity Resolution

A mix of person and household scoped ID profiles are joined to form identity clusters as described in detail here.

Last updated 29 days ago