Kafka Topic Naming Conventions That Scale

Build a Kafka topic naming convention that scales. Structural patterns, ACL alignment, and common mistakes that break governance.

Stéphane DerosiauxStéphane Derosiaux · July 13, 2024 ·
Kafka Topic Naming Conventions That Scale

Topic naming feels like bikeshedding until you have 500 topics and can't find anything. The convention you choose becomes permanent infrastructure—Kafka topics cannot be renamed.

I've seen organizations stuck with OrderEvents, order_events, order-events, and ORDEREVENTS across 20 teams. Nobody can find anything. ACLs require one entry per topic.

We started with ad-hoc names. Five years later, we have 800 topics and no consistent way to search or apply ACLs. A proper naming convention would have saved hundreds of hours.

Platform Lead at a logistics company

A Pattern That Works

The most battle-tested pattern uses hierarchical segments:

<domain>.<entity>.<event>

Examples:

orders.order.created
orders.order.cancelled
inventory.item.reserved
payments.transaction.completed

This enables wildcard ACLs:

# Grant service access to all order topics with one rule
kafka-acls.sh --bootstrap-server localhost:9092 \
  --add --allow-principal User:order-service \
  --operation Read --operation Describe \
  --topic orders. --resource-pattern-type prefixed

At 500 topics, consistent prefixes mean ~10 ACL entries instead of 500.

Pick One Separator

Kafka warns about mixing . and _ because they collide in metric names:

kafka-topics.sh --create --topic "test.naming" ...
# WARNING: topics with '.' or '_' could collide

Pick one and enforce it. Most organizations choose . for hierarchy, - for word separation: orders.order-created.

Aligning With ACLs

Naming PatternACL Complexity
orders.order.created1 prefix ACL per domain
OrderCreated, order_created1 ACL per topic

Common Mistakes

Naming after teams:

checkout-team.order-events  # Bad
orders.order.created        # Good

Teams rename. Services get rewritten. The topic stays forever with a misleading name.

Naming after consumers:

analytics-orders            # Bad

A topic describes data, not who reads it.

Environment prefixes:

prod.orders.order.created   # Bad

Use separate clusters for environments instead.

Enforcing Conventions

# server.properties - Force controlled topic creation
auto.create.topics.enable=false

Beyond configuration, a self-service portal can enforce naming conventions automatically when teams request new topics.

Define topics in source control and validate in CI:

# topics.yaml
topics:
  - name: orders.order.created
    partitions: 6
    replication: 3

Your pipeline validates name matches ^[a-z]+\.[a-z-]+\.[a-z]+$.

Kafka Streams Internal Topics

Streams creates internal topics automatically:

my-app-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog

If you change your topology, these names change. Fix: Name operators explicitly:

stream.groupByKey(Grouped.as("orders-by-customer"))
      .aggregate(..., Materialized.as("order-counts"));

The organizations running Kafka most efficiently invested in naming conventions early. The ones struggling grew into whatever pattern emerged from ad-hoc decisions.

Book a demo to see how Conduktor Console provides centralized topic governance with naming validation and policy enforcement.