📨 Apache Kafka + Makoto Integration Concept

Producer/Consumer Interceptors — stream-window attestations per partition.

Note: This page explores how Makoto Levels could be implemented on Apache Kafka. It is a conceptual integration proposal — illustrative, not a shipped library. The patterns shown use real Apache Kafka APIs; the Makoto pieces are sketches you (or we) could build out.

What is Apache Kafka?

Apache Kafka is the central nervous system of most data platforms. Producers append records to topics; consumers pull them. Both interfaces support Interceptors — pluggable hooks Makoto can use to emit attestations without touching application code.

Producer InterceptorHash + sign records on the way in
Consumer InterceptorVerify DBOM headers on the way out
Stream Window AttestationPer-N-records or per-time-window summaries
Schema Registry HookSchema ID becomes part of the attestation predicate

Integration Approach

Primary pattern: ProducerInterceptor + ConsumerInterceptor + header conventions. Below are the integration options ordered by lift required.

How Makoto attaches to Apache Kafka

  • makoto-kafka client lib — Drop-in `MakotoProducer` / `MakotoConsumer` subclasses for the Java/Python/Go clients.
  • Connect SMT — Single Message Transform plugin for Kafka Connect — adds DBOM headers to every record in flight.
  • Kafka Streams DSL — `.via(MakotoTransformer.attest())` step that fits cleanly into existing topologies.
  • MirrorMaker 2 hook — Cross-cluster replication carries attestations across regions.

Conceptual Code Example

Concept: Kafka Connect SMT adds attestation headers

No application changes — works for any source/sink connector

# connect/source-orders.json
name: orders-postgres-source
config:
  connector.class: io.debezium.connector.postgresql.PostgresConnector
  database.hostname: pg.internal
  database.dbname: sales
  topic.prefix: orders

  # Makoto Single Message Transform
  transforms: makoto
  transforms.makoto.type: dev.makoto.kafka.AttestationSMT
  transforms.makoto.level: 2
  transforms.makoto.signing.key.id: KMS_KEY_ARN
  transforms.makoto.window.records: 1000
  transforms.makoto.dbom.topic: makoto.dbom

  # Each record gets headers:
  #   makoto.content_hash = sha256 of value
  #   makoto.window_id    = monotonic window counter
  #   makoto.signature    = signed window DBOM ref
  #
  # Every 1000 records, a Stream-window attestation is
  # published to the makoto.dbom topic with the window
  # boundary digests, record count, and signing key id.

Potential Use Cases

Financial Tick Streams

Every market tick window carries a signed attestation — replay any time bucket end-to-end.

IoT Telemetry

Edge producers emit DBOMs proving the firmware that produced each reading.

CDC Replication

Debezium → Kafka → warehouse keeps a verifiable chain across the boundary.

Cross-org Data Sharing

Partners only consume topics whose interceptors verify a trusted issuer key.

Interested in Apache Kafka + Makoto?

This is a conceptual integration. If you're shipping Apache Kafka pipelines and want to add Makoto attestations, open an issue or reach out — we'd love to scope a real implementation.

Learn about Apache Kafka Read Makoto Spec All Integrations