How Conduktor fits the Kafka you already run
A management plane and inline proxies in front of the clusters you already run. Self-hosted, Kafka-protocol compliant, nothing to migrate.
Self-hosted, in your environment
Runs inside your infrastructure: cloud, hybrid, on-prem, or air-gapped. Your data and credentials never leave your network. It connects to the Kafka, identity, and key systems you already run.
Architecture
Console is the management plane: control, governance, automation, and insights, reached through a UI, API, CLI, and MCP. The data plane runs two proxies on the wire: the Gateway for Kafka and a separate Schema Registry Proxy for schemas, routing, securing, and enforcing policy inline. Your clients keep speaking Kafka, seamlessly, with no SDK swap or rewrite.
The four components
Console is the management plane; the Gateway and Schema Registry proxies are the data plane. Observability handles metrics and alerting, and the CLI and MCP open everything to automation and AI.
Control, governance, automation, and insights: Web UI, API, RBAC, policies, audit, GitOps, Terraform. One container plus PostgreSQL; two or more for HA.
Two inline proxies, stateless and horizontally scaled. The Gateway (Kafka protocol) routes and enforces encryption, masking, data quality, quotas, and multi-tenancy; a separate Schema Registry Proxy handles schema authentication and authorization.
Real-time metrics and self-service alerts for platform and app teams on their own topics and consumer groups. Built-in alerts and Insights flag Kafka best-practice risks. Bundled, or bring your own Prometheus or Mimir.
Resources as code via the CLI and Terraform; AI agents via the built-in MCP server and Skills, all under the same RBAC.
Where Gateway sits in your data path
A transparent, Kafka-protocol proxy. Clients repoint their bootstrap server, no rewrite. Records pass through untouched unless a policy encrypts, masks, or rejects them. A separate Schema Registry Proxy fronts your registry the same way. One network hop, minimal latency, the same control model across any provider. See how Gateway cuts Kafka DR from hours to minutes.
No single point of failure
Stateless: the Kafka cluster holds all state. Run several instances; clients reconnect to a healthy one automatically. A restart never drops traffic.
Load balancing & HA →Scales horizontally
CPU-bound: encryption, masking, and guardrails run on the wire. ~20–30 MB/s per instance (2 CPU / 4 GB), more on bigger nodes. Add instances as traffic and policies grow; three or more in production.
Scaling Gateway →Disaster-recovery failover
Hot-switch traffic to a secondary Kafka cluster through the Gateway API, with no client changes.
Failover guide →Where it runs, and what it needs
Kubernetes with Helm (recommended), Docker, or AWS. Production needs an external PostgreSQL 13+, blob storage (S3, GCS, or Azure), and Kafka 2.7+. The public reference architecture covers HA, sizing, and a runnable example.
Plugs into your security stack
Conduktor enforces controls using the identity and key systems you already run, not new ones it asks you to adopt. See achieving data security for Kafka.
SSO over OIDC (Okta, Entra ID, Google, Keycloak, and more) or LDAP, with group mapping. Keys stay in your own Vault, AWS KMS, or Azure Key Vault.
Field-level or full-payload encryption at the proxy, across JSON, Avro, and Protobuf. Masking hides sensitive fields without touching the data in Kafka.
Granular, prefix-aware RBAC across topics, groups, subjects, connectors, and clusters. Teams self-serve within the policies you set.
Audit logs in the vendor-neutral CloudEvents format. Virtual Clusters and Partner Zones isolate teams, environments, and partners on one cluster.
Deep dives for architects
Go deeper on the decisions behind the architecture.
Is Gateway in the data path, and what happens if it fails?
Yes, it's an inline, stateless proxy. The Kafka cluster holds all state, so run several instances and clients reconnect on standard retry. A restart never drops traffic; run three or more in production.
Do I need to change my applications?
No. Gateway speaks the Kafka protocol. Point your bootstrap server at it and keep your standard clients: no SDK swap, no rewrite. Records pass through unless a policy applies.
Does my data ever leave my environment?
No. Conduktor is self-hosted; your messages, credentials, and keys stay in your network. Even the built-in MCP server runs locally, under your RBAC.
Which clouds and Kafka providers are supported?
AWS, Azure, GCP, or on-prem, in front of Confluent, Amazon MSK, Aiven, Redpanda, and open-source Apache Kafka (2.7+). Kubernetes with Helm recommended; Docker and AWS also supported.
Can we integrate with Vault or a KMS?
Yes. Keys live in HashiCorp Vault, AWS KMS, or Azure Key Vault, bring-your-own. Conduktor never holds them.
Can it run fully air-gapped?
Yes. Offline installs with local registries, updated without external network access.
Planning your deployment?
From sizing and high availability to compliance, our team will walk through the architecture and the trade-offs with you.