Back to all guides
Environment

Data Residency & Composite Databases

Control where your Identity Knowledge Graph data lives: regions, replicas, routing data to specific locations with a composite database, and location-aware authorization with 3.0-kbac policies.

What is Data Residency?

Data residency controls where your Identity Knowledge Graph (IKG) physically lives. Every IndyKite project is created in a region, and that region determines where your graph data is stored and processed.

This matters when your data is subject to regulatory requirements (for example GDPR or sector-specific rules that mandate in-region storage), or when you want the IKG close to your users for latency reasons.

Which residency settings does a project have?

Field Description Values
region Region where the project's IKG is provisioned. Required at creation, cannot be changed afterwards. Depends on your environment, for example europe-west1, us-east1
ikg_size Storage size of the IKG instance; each size comes with a corresponding CPU allocation. Depends on your environment, for example 2GB, 4GB, 8GB, 16GB, 32GB, 64GB, 128GB, 192GB, 256GB, 384GB, 512GB
replica_region Optional second region that receives a synchronized read replica of the IKG. Depends on your environment, for example europe-west1, us-east1, us-west1
ikg_status Provisioning state of the IKG (read-only, returned on reads). PENDING, ACTIVE, FAILED, PAUSED

How do replicas work?

Setting replica_region provisions a second IKG in another region that is continuously populated from the primary by change-data-capture replication.

  • The replica must be in a different region than the primary, but on the same geographical continent.
  • The replica is read-only: it is served through a read-only database user and cannot accept writes.
  • Replication is one-way, from the primary region to the replica region.
  • Use it for in-region read access (queries, authorization decisions) close to users in a second geography.

What is a Composite Database?

A region places your whole IKG in one location. A composite database goes further: it lets one logical IKG span multiple physical databases, so that individual nodes can be stored in a specific location: for example, EU customer records in an EU database and US customer records in a US database, while your applications keep talking to a single graph.

A composite IKG is made of constituent databases:

Constituent What it stores
Global database Lightweight proxy nodes (external ID, type, and location only: no property data) for every located node, plus the relationships that connect nodes across locations.
Location databases The full node data for each logical location (for example east, west).
Default database Nodes ingested without a location: they follow the same path as a regular, non-composite IKG.

Two settings on the project's database connection wire this together:

  • composite_db_name: the name of the composite database. Leave it empty for a regular single-database IKG.
  • alias_mapping: a URL-query-encoded map from logical location names to constituent databases, for example global=db1&east=db2&west=db3. The location names on the left are the values you will later put in the location field of Capture API requests.

When should I use a composite database?

  • You must keep certain nodes' data in a specific jurisdiction, but still need one connected graph across all of them (cross-location relationships, single authorization model).
  • You want per-record placement decisions (this person's data stays in the EU) rather than a per-project region choice.

Availability: composite databases are supported for customer-hosted IKGs only: you run your own Neo4j deployment and IndyKite connects to it. IndyKite-managed projects use a regular single-database IKG in the project's region.

How do I create a project with residency settings?

Residency is set on the Config API when creating a project. Authenticate with your Service Account token (see the credentials guide).

POST /configs/v1/projects

{
  "name": "my-project",
  "display_name": "My Project",
  "organization_id": "gid:your-organization-id",
  "region": "us-east1",
  "ikg_size": "2GB",
  "replica_region": "us-west1"
}

With header Authorization: Bearer <service-account-token>.

Provisioning is asynchronous: the response returns immediately, while the IKG is created in the background. Poll the project until it is ready:

GET /configs/v1/projects/{id}

{
  "id": "gid:your-project-id",
  "region": "us-east1",
  "ikg_size": "2GB",
  "replica_region": "us-west1",
  "ikg_status": "ACTIVE"
}

Wait for ikg_status to become ACTIVE before ingesting data. PENDING means provisioning is still running; FAILED means it did not complete.

How do I set up a composite database?

Step 1: Create the databases in your Neo4j deployment

The composite database and its constituents must exist before you create the project. IndyKite connects to them and configures routing; it does not create the databases for you. On your Neo4j instance:


CREATE DATABASE db1 IF NOT EXISTS;
CREATE DATABASE db2 IF NOT EXISTS;
CREATE DATABASE db3 IF NOT EXISTS;
CREATE COMPOSITE DATABASE ikcomposite IF NOT EXISTS;
CREATE ALIAS ikcomposite.db1 FOR DATABASE db1;
CREATE ALIAS ikcomposite.db2 FOR DATABASE db2;
CREATE ALIAS ikcomposite.db3 FOR DATABASE db3;

In a real deployment the constituent databases typically live on servers in different locations; the composite database presents them as one graph.

Step 2: Create the project with the composite connection

POST /configs/v1/projects

{
  "name": "my-project",
  "display_name": "My Project",
  "organization_id": "gid:your-organization-id",
  "region": "europe-west1",
  "db_connection": {
    "url": "neo4j://your-neo4j-host:7687",
    "username": "neo4j",
    "password": "<your-password>",
    "name": "db1",
    "composite_db_name": "ikcomposite",
    "alias_mapping": "global=db1&east=db2&west=db3"
  }
}
db_connection field Meaning
url, username, password Connection to your Neo4j deployment (required).
name The default constituent database: used for operations that carry no location.
composite_db_name Name of the composite database created in Step 1. Empty means a regular IKG.
alias_mapping Location-to-constituent map. By convention, map the global location to the constituent that should hold proxy nodes and cross-location relationships.

Step 3: Wait for the IKG to become active

As with any project, poll GET /configs/v1/projects/{id} until ikg_status is ACTIVE. The response echoes db_connection.composite_db_name and db_connection.alias_mapping so you can verify the configuration round-trips.

Can I change the composite configuration of an existing project?

Yes, with an update (subject to a few rules):

PUT /configs/v1/projects/{id}

{
  "db_connection": {
    "url": "neo4j://your-neo4j-host:7687",
    "username": "neo4j",
    "password": "<your-password>",
    "name": "db1",
    "composite_db_name": "ikcomposite",
    "alias_mapping": "global=db1&east=db2&west=db3"
  }
}
  • Optionally send an If-Match: "<etag>" header (from a previous read) to guard against concurrent updates.
  • db_connection can only be updated on customer-hosted projects; on IndyKite-managed projects it is rejected. Omitting db_connection leaves the existing connection untouched.
  • The databases referenced by the new mapping must already exist in Neo4j. The update rewires configuration and routing: it does not migrate existing data between constituent databases.
  • The hosting type itself cannot change: an IndyKite-managed project cannot be turned into a composite one. Composite requires customer-hosted from creation.
  • The update is applied asynchronously: poll ikg_status as for creation.

How do I ingest data with locations?

On a composite IKG, the Capture API routes data by location. Authenticate with your Application Agent key: X-IK-ClientKey: <token-from-credentials-file>.

  • Nodes are routed per node with an optional location field (2–32 characters, must be a key of the project's alias_mapping).
  • Relationships are routed per request with "use_global_db": true: relationships can connect nodes living in different locations, so they are stored in the global constituent alongside the proxy nodes.

Upsert nodes with per-node locations

POST /capture/v1/nodes

{
  "nodes": [
    {
      "external_id": "person-alice",
      "type": "Person",
      "is_identity": true,
      "location": "east",
      "properties": [
        { "type": "name", "value": "Alice Marchetti" },
        { "type": "email", "value": "alice@example.com" }
      ]
    },
    {
      "external_id": "person-karel",
      "type": "Person",
      "is_identity": true,
      "location": "west",
      "properties": [
        { "type": "name", "value": "Karel Plihal" }
      ]
    },
    {
      "external_id": "car-kitt",
      "type": "Car",
      "is_identity": false,
      "properties": [
        { "type": "manufacturer", "value": "Pontiac" }
      ]
    }
  ]
}

What happens:

  • The full person-alice node is stored in the east constituent, person-karel in west.
  • For every located node, a proxy node (external ID, type, and location (no properties)) is automatically upserted into the global constituent, so the nodes stay addressable graph-wide.
  • car-kitt has no location, so it goes to the default database, exactly as on a non-composite IKG.

Upsert relationships into the global database

POST /capture/v1/relationships

{
  "use_global_db": true,
  "relationships": [
    {
      "source": { "external_id": "person-alice", "type": "Person" },
      "target": { "external_id": "car-kitt", "type": "Car" },
      "type": "OWNS",
      "properties": [ { "type": "status", "value": "active" } ]
    }
  ]
}

The relationship is created in the global constituent between the proxy nodes, connecting entities whose full data lives in different locations.

Which delete operations are location-aware?

All Capture API delete endpoints accept the same routing fields:

Endpoint Routing field
POST /capture/v1/nodes/delete location per node
POST /capture/v1/nodes/properties/delete location per node
POST /capture/v1/nodes/properties/metadata/delete location per node
POST /capture/v1/relationships/delete use_global_db per request
POST /capture/v1/relationships/properties/delete use_global_db per request

Example: deleting nodes across locations (deleting a located node removes both its data node in the location constituent and its proxy in the global database):

POST /capture/v1/nodes/delete

{
  "nodes": [
    { "external_id": "person-alice", "type": "Person", "location": "east" },
    { "external_id": "person-karel", "type": "Person", "location": "west" },
    { "external_id": "car-kitt", "type": "Car" }
  ]
}

What happens if the location is wrong?

Situation Result
A location that is not a key in the project's alias_mapping Request fails: unknown location
A location on a project with no composite database configured Request fails: composite database not configured
No location at all Node goes to the default database (legacy behavior, always safe)

How do queries and authorization work on a composite IKG?

Reads do not fan out across constituent databases by default. ContX IQ queries (POST /contx-iq/v1/execute) and KBAC / AuthZEN authorization checks with 2.0-kbac policies (POST /access/v1/*) execute against the default constituent database only: the one named by db_connection.name.

  • Point the default at the global constituent (as in this guide's examples, where name is the database mapped to global). Authorization decisions that depend on graph structure keep working, because they traverse the proxy nodes and the cross-location relationships stored there.
  • Located property data is out of read scope. Proxy nodes carry only external ID, type, and location: the full properties stored in a location constituent (for example a person's email in east) are not returned by queries and cannot be referenced in policy conditions running against the default database.
  • CIQ and 2.0-kbac policy Cypher cannot route to a constituent. A USE clause or a CALL { } subquery is rejected when the policy is created (see the CIQ Cypher guide).
  • To make authorization decisions against a location constituent, author the policy as 3.0-kbac: the next section shows how.

How does authorization work with data residency? (3.0-kbac)

The 3.0-kbac policy version makes KBAC / AuthZEN authorization location-aware. Where the platform rewrites a 2.0-kbac condition and always runs it against the default database, a 3.0-kbac condition is raw Cypher: it runs as you wrote it, and you author the composite-database routing yourself with USE graph.byName(...). Residency support is opt-in per policy: existing 2.0-kbac policies are unchanged.

Routing can be:

  • Static: USE graph.byName('ikcomposite.db2'): always evaluates in that constituent.
  • Dynamic: USE graph.byName($region): $region becomes a location parameter. The caller passes a logical location (a key of the project's alias_mapping, for example "east") in the AuthZEN request's context.input_params, and IndyKite translates it to the physical constituent just before execution. Callers never see or supply physical database names.

Example: a location-routed policy

Create the policy on the Config API (POST /configs/v1/authorization-policies) like any other KBAC policy; only the policy JSON differs:


{
  "meta": { "policy_version": "3.0-kbac" },
  "subject": { "type": "Person" },
  "actions": ["CAN_DRIVE"],
  "resource": { "type": "Car" },
  "condition": {
    "cypher": "USE graph.byName($region) MATCH (subject:Person)-[:OWNS]->(resource:Car)"
  }
}

Then evaluate with a location in context.input_params:

POST /access/v1/evaluation

{
  "subject": { "type": "Person", "id": "person-alice" },
  "action": { "name": "CAN_DRIVE" },
  "resource": { "type": "Car", "id": "car-kitt" },
  "context": { "input_params": { "region": "east" } }
}

"east" is the logical location from alias_mapping, not a Neo4j database name. The same pattern works on all AuthZEN endpoints: single evaluation, batch evaluations, and the three search endpoints.

What can 3.0-kbac Cypher do that 2.0-kbac cannot?

  • USE graph.byName(...) clauses, top-level or per CALL { } subquery, so one condition can combine matches from several constituents.
  • CALL { } subqueries with inner RETURNs (both rejected in 2.0-kbac).
  • Mutating clauses (CREATE, MERGE, SET, DELETE, and so on) remain blocked, as does a top-level RETURN: the platform appends the projection itself.

Authoring rules for 3.0-kbac

  • The condition must still bind subject and resource variables; the platform pins them by type and external ID and replaces the final projection.
  • The graph.byName() argument must be a string literal or a single parameter: expressions like coalesce($region, 'eu') are rejected at creation.
  • A routing parameter cannot be referenced anywhere else in the Cypher: its value is rewritten to the physical alias at request time.
  • $subject_external_id, $subject_type, $resource_external_id, and $resource_type are bound by the platform: never supply them in input_params or use them as routing parameters.
  • $subject_id must not be referenced at all: on a composite IKG the same logical subject has a different internal node ID per location, so it is rejected at creation.

What happens if the location is wrong at decision time?

Situation Result
Required location parameter missing from context.input_params, or not a non-empty string 422 Unprocessable Entity
Location is not a key of the project's alias_mapping 422 Unprocessable Entity (unknown location)
Policy uses a location parameter but the project has no composite database 422 Unprocessable Entity (requires a composite database)
Bearer-token call where the token's subject differs from the requested subject 403 Forbidden (permission denied)

See the AuthZEN guide for the full request shape, and the authz-7 / authz-8 resources for end-to-end examples.

Related resources