Kafka Outbound Events / Signal - Capture

Steps to create the Outbound Events configuration in a project with Kafka as provider in Confluent.

Use case

One Outbound Events configuration can be set per project.

The configuration will send events to the defined providers through the designated routes each time a modification is made in the stipulated IKG section.

In this use case, a configuration is created to send a message each time an upsert or a delete action is executed on any node with a Car label and a property manufacturer with value pontiac.

Requirements

- ServiceAccount credentials created in the IndyKite Hub for your organization.

- AppAgent credentials created in the IndyKite Hub, using the REST endpoints or using Terraform for your Project / Application.

- A valid API Key for a Confluent environment.

Steps

1. Create a topic on Confluent.

2. Create an Outbound Events configuration with indykite.audit.capture.* eventType and key pairs in a route for a Confluent provider.

3. Ingest data with some node with a Car label and a property manufacturer with value pontiac, in the IndyKite platform through the hub or the REST endpoint.

4. Add other nodes with a Car label and a property manufacturer with value pontiac.

5. Check that messages are received in the topic each time an upsert or a delete action is executed on any node with a Car label and a property manufacturer with value pontiac.

6. Check that events related to nodes with other properties are not sent to the topic.

Step 2

main.tf


terraform {
  required_providers {
    indykite = {
      source  = "indykite/indykite"
      version = 1.26. # or latest version
    }
  }
}

# indykite provider integrates IndyKite platform with Terraform scripting.
# Provider for now does not support any parameters and all is set within service account credential file.
provider "indykite" {}
resource "time_static" "example" {}

resource "indykite_event_sink" "outbound_event" {
  name         = "outbound-event"
  display_name = "Outbound Event"
  location     = "project_id"
  providers {
    provider_name = "provider-name"
    kafka {
      brokers  = ["broker"]
      topic    = "topic_signal"
      username = "api_key"
      password = "api_secret"
    }
  }
  routes {
    provider_id     = "provider-name"
    stop_processing = true
     keys_values_filter {
      key_value_pairs {
        key   = "manufacturer"
        value = "pontiac"
      }
       key_value_pairs {
        key   = "captureLabel"
        value = "Car"
      }
      event_type = "indykite.audit.capture.upsert.node"
    }
    route_display_name = "Configuration Audit Events"
    route_id           = "config-audit-log"
  }
}

Step 3

Ingest the nodes needed for this use case.

POST https://eu.api.indykite.com/capture/v1/nodes/

{
  "nodes": [
    {
      "external_id": "alice",
      "is_identity": true,
      "type": "Person",
      "properties": [
        {
          "type": "email",
          "value": "alice@email.com"
        },
        {
          "type": "given_name",
          "value": "Alice"
        },
        {
          "type": "last_name",
          "value": "Smith"
        }
      ]
    },
    {
      "external_id": "knightrider",
      "type": "Person",
      "is_identity": true,
      "properties": [
        {
          "type": "email",
          "value": "knightrider@demo.com"
        },
        {
          "type": "name",
          "value": "Michael Knight"
        }
      ]
    },
    {
      "external_id": "satchmo",
      "type": "Person",
      "is_identity": true,
      "properties": [
        {
          "type": "email",
          "value": "satchmo@demo.com"
        },
        {
          "type": "name",
          "value": "Louis Armstrong"
        }
      ]
    },
    {
      "external_id": "karel",
      "type": "Person",
      "is_identity": true,
      "properties": [
        {
          "type": "email",
          "value": "karel@demo.com"
        },
        {
          "type": "name",
          "value": "Karel Plihal"
        }
      ]
    },
    {
      "external_id": "kitt",
      "type": "Car",
      "is_identity": false,
      "properties": [
        {
          "type": "manufacturer",
          "value": "pontiac"
        },
        {
          "type": "model",
          "value": "Firebird"
        }
      ]
    },
    {
      "external_id": "cadillacv16",
      "type": "Car",
      "is_identity": false,
      "properties": [
        {
          "type": "manufacturer",
          "value": "Cadillac"
        },
        {
          "type": "model",
          "value": "V-16"
        }
      ]
    },
    {
      "external_id": "harmonika",
      "type": "Bus",
      "is_identity": false,
      "properties": [
        {
          "type": "manufacturer",
          "value": "Ikarus"
        },
        {
          "type": "model",
          "value": "280"
        }
      ]
    },
    {
      "external_id": "listek",
      "type": "Ticket",
      "is_identity": false
    },
    {
      "external_id": "airbook-xyz",
      "type": "Laptop",
      "is_identity": false
    }
  ]
}

Step 4

Ingest the nodes needed for this use case.

POST https://eu.api.indykite.com/capture/v1/nodes/

{
  "nodes": [
    {
      "external_id": "kitten",
      "type": "Car",
      "is_identity": false,
      "properties": [
        {
          "type": "manufacturer",
          "value": "pontiac"
        },
        {
          "type": "model",
          "value": "Bonneville"
        }
      ]
    },
    {
      "external_id": "kitty",
      "type": "Car",
      "is_identity": false,
      "properties": [
        {
          "type": "manufacturer",
          "value": "pontiac"
        },
        {
          "type": "model",
          "value": "Catalina"
        }
      ]
    }
  ]
}