# Kafka

Connect to any Apache Kafka cluster — browse topics, consumer groups, and cluster metadata.

## What you can manage

- **Cluster** — broker count, controller, cluster ID, version.
- **Topics** — list, create (with partitions + replication factor), delete.
- **Consumer groups** — list, inspect protocol, delete.

## Credentials

Infrawrench uses a custom `kafka://` URL to bundle the bootstrap brokers and authentication into one credential field.

### Plaintext (no auth)

```
kafka://broker1:9092
```

For multiple brokers, use the `brokers=` query param since standard URLs can't carry a comma-separated host:

```
kafka://placeholder:9092?brokers=b1:9092,b2:9092,b3:9092
```

### SASL / SSL

Supported SASL mechanisms: `plain`, `scram-sha-256`, `scram-sha-512`.

```
kafka://broker1:9092?sasl=scram-sha-256&user=alice&password=…&ssl=true
```

Equivalent shorthand — credentials in the URL userinfo, `kafkas://` for TLS:

```
kafkas://alice:secret@broker1:9092?sasl=scram-sha-256
```

URL-encode special characters in `user` and `password` (e.g. `@` → `%40`).

### Reference an output

If you've added a managed-Kafka resource elsewhere (Aiven, Confluent Cloud, MSK), reference its connection-string output instead of pasting a URL.

![Kafka Add-account form with the kafka:// URL field filled in](https://agent-assets.infrawrench.com/docs-screenshots/plugins/kafka/add-account.png)

## Notable flows

- **Sidebar lists Topics and Consumer Groups** under the cluster — the cluster itself is the top-level entry per account.
- **Create topic** from the cluster's detail view — pick partition count and replication factor.
- **Create topic from a peer pane** — when Kafka is surfaced as a peer of another resource (e.g. a DigitalOcean Managed Database's **Kafka** tab), the Topics group shows a **+ Create Topic** button even when the cluster is empty, so a fresh cluster isn't a dead-end. Consumer groups have no create button — they form when a consumer subscribes.
- **Delete topic / group** from the resource's own detail view.
- **Produce a test record** on any topic from the **Produce** tab on its detail page — see [Send test messages](../features/send-test-message.md). The connection's credentials need produce ACLs on the topic.

![Cluster detail view showing brokers, topics, and consumer groups](https://agent-assets.infrawrench.com/docs-screenshots/plugins/kafka/cluster-detail.png)

## Tips & limits

- Internal topics that start with `__` (e.g. `__consumer_offsets`) are hidden from the topic list.
- Replication factor must be `≤` the number of brokers — creation will fail upstream otherwise.
- For private clusters (AWS MSK in a VPC, on-prem brokers), bind the account to a bastion so the host tunnels Kafka traffic through it.
- This plugin uses [kafkajs](https://kafka.js.org/); native-binary clients (librdkafka-backed) are not currently supported.