How to Secure Kafka with SSL
slug
how-to-secure-kafka-with-ssl
I’ll never forget the day a client called me at 2 AM. Their Kafka cluster — processing 50,000 events per second — had been breached. An internal team had left TLS disabled “for testing.” Three months later, a rogue script started dumping customer PII into a public bucket. They lost the deal. They almost lost the company.
That call changed how I think about encryption. SSL/TLS isn’t just a checkbox on a compliance form. It’s the difference between “we’ll patch it next sprint” and “we’re in the news for the wrong reason.”
In this guide I’ll walk you through exactly how to secure Kafka with SSL — from generating certificates to configuring producers, consumers, and brokers. I’ll share the gotchas I’ve hit in production, the commands that actually work, and the common mistakes that get teams in trouble. We’ll also touch on how this fits into broader data infrastructure decisions — like when you’re comparing Kafka vs Pulsar vs RabbitMQ vs NATS for your use case.
If you’re running Kafka in 2026 and you don’t have SSL enforced on every listener, stop reading and go fix it. Then come back.
Why SSL Matters More Than Ever
Most people think securing Kafka is about firewalls and network segmentation. They’re not wrong — but they’re missing the point. Kafka’s default configuration sends data in plaintext. If I can reach your broker port (9092), I can read every message. No authentication, no encryption.
SSL solves two problems at once:
- Encryption in transit — nobody can sniff your data between producer and broker, or broker and consumer.
- Authentication — SSL certificates prove the identity of both client and server.
In 2026, with regulations like GDPR, CCPA, and India’s DPDP Act getting teeth, plaintext Kafka is a legal liability. I’ve seen startups blow seed rounds because their SOC 2 auditor found Kafka traffic unencrypted. Don’t be that startup.
Also — SSL isn’t just for external traffic. Encrypt everything. Internal traffic between data centers, between microservices, even between Kafka brokers themselves. The “trust the network” mindset is dead.
Before You Start: The Certificate Lifecycle
You need three things:
- A Certificate Authority (CA) — self-signed or internal (e.g., using OpenSSL or HashiCorp Vault).
- A server certificate for each broker, signed by that CA.
- A client certificate (or truststore) for each producer/consumer.
Never use the same certificate for server and client. I know it’s tempting. Don’t.
Here’s the stack I use at SIVARO for most deployments:
CA: /etc/kafka/ssl/ca-cert.pem (private key kept offline)
Broker cert: /etc/kafka/ssl/kafka-server-certs/
Client truststore: /etc/kafka/ssl/client-truststore.jks
We generate everything with OpenSSL. No Java keytool unless we need JKS format for old clients. If you can, use PEM everywhere — it’s simpler, more transparent, and easier to debug.
Step 1: Generate the CA
bash
# Create a private key for the CA
openssl genrsa -out ca-key.pem 4096
# Self-signed CA certificate (10 years)
openssl req -x509 -new -nodes -key ca-key.pem -sha256 -days 3650 -out ca-cert.pem -subj "/C=US/ST=NY/O=MyCompany/CN=Kafka-CA"
Store ca-key.pem offline. If that leaks, your entire PKI is compromised.
Step 2: Generate Broker Certificates
Each broker needs a certificate with the SAN (Subject Alternative Name) matching its hostname. Wildcards work, but I prefer explicit SANs for auditability.
bash
# Generate broker key and CSR
openssl genrsa -out broker-key.pem 4096
openssl req -new -key broker-key.pem -out broker.csr -subj "/C=US/ST=NY/O=MyCompany/CN=kafka-broker-1.internal"
# Sign with CA
openssl x509 -req -in broker.csr -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out broker-cert.pem -days 365 -sha256 -extfile <(printf "subjectAltName=DNS:kafka-broker-1.internal,IP:10.0.1.10")
Repeat for each broker. I’ve seen people reuse the same cert for 10 brokers. That works. It also means if you revoke one, you revoke all. Split them.
Step 3: Configure Kafka Brokers for SSL
Kafka’s server.properties needs three listeners usually: one for inter-broker, one for clients, and one for controller (in KRaft mode). Enable SSL on each.
properties
# server.properties (Kafka 3.x+)
listeners=PLAINTEXT://0.0.0.0:9092,SSL://0.0.0.0:9093,CONTROLLER://0.0.0.0:9094
advertised.listeners=PLAINTEXT://kafka-broker-1.internal:9092,SSL://kafka-broker-1.internal:9093
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,CONTROLLER:SSL
ssl.keystore.location=/etc/kafka/ssl/keystore.jks
ssl.keystore.password=changeit
ssl.key.password=changeit
ssl.truststore.location=/etc/kafka/ssl/truststore.jks
ssl.truststore.password=changeit
ssl.client.auth=required # Forces mutual TLS
ssl.endpoint.identification.algorithm=HTTPS # Verify hostname
Yes, I’m using JKS here because Kafka’s Java client expects it. If you want to use PEM directly, you can – but only in Kafka 3.4+ with ssl.keystore.type=PEM. Most shops still use JKS.
The ssl.client.auth=required line is critical. It means every client must present a valid certificate signed by your CA. Without this, SSL only encrypts — it doesn’t authenticate.
Step 4: Convert PEM to JKS (If You Must)
bash
# Create PKCS12 from broker cert and key
openssl pkcs12 -export -in broker-cert.pem -inkey broker-key.pem -out broker.p12 -name kafka-broker -CAfile ca-cert.pem -caname root
# Import into JKS keystore
keytool -importkeystore -deststorepass changeit -destkeypass changeit -destkeystore keystore.jks -srckeystore broker.p12 -srcstoretype PKCS12 -srcstorepass changeit -alias kafka-broker
# Import CA into truststore
keytool -import -trustcacerts -alias ca -file ca-cert.pem -keystore truststore.jks -storepass changeit -noprompt
Do this once. Don’t script it to run every day. I’ve seen cron jobs corrupt keystores because of missing inputs.
Step 5: Configure Producers and Consumers
On the client side, you need to trust the broker’s certificate (the CA) and present your own client certificate.
java
Properties props = new Properties();
props.put("bootstrap.servers", "kafka-broker-1.internal:9093");
props.put("security.protocol", "SSL");
props.put("ssl.truststore.location", "/etc/kafka/client-truststore.jks");
props.put("ssl.truststore.password", "changeit");
props.put("ssl.keystore.location", "/etc/kafka/client-keystore.jks");
props.put("ssl.keystore.password", "changeit");
props.put("ssl.key.password", "changeit");
props.put("ssl.endpoint.identification.algorithm", "HTTPS");
KafkaProducer<String, String> producer = new KafkaProducer<>(props);
For Python (confluent-kafka):
python
from confluent_kafka import Producer
conf = {
'bootstrap.servers': 'kafka-broker-1.internal:9093',
'security.protocol': 'SSL',
'ssl.ca.location': '/etc/kafka/ca-cert.pem',
'ssl.certificate.location': '/etc/kafka/client-cert.pem',
'ssl.key.location': '/etc/kafka/client-key.pem',
'ssl.endpoint.identification.algorithm': 'https',
}
producer = Producer(conf)
One gotcha: if your client doesn’t send a certificate, and you set ssl.client.auth=required on the broker, the connection fails with a cryptic SSL handshake error. Debugging that takes hours. I speak from experience.
Step 6: Enforce SSL Everywhere
Don’t leave the PLAINTEXT listener enabled in production. I get it – you need it for monitoring or legacy tools. Here’s the contrarian take: disable it anyway. If your monitoring tool doesn’t support SSL in 2026, replace it.
In Kafka 3.0+, you can use authorizer.class.name to block PLAINTEXT connections even if the listener is open:
properties
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
allow.everyone.if.no.acl.found=false
Then define ACLs on the SSL listener only. This is belt-and-suspenders. Use it.
Common Mistakes (and How to Avoid Them)
1. Using the Same Keystore for Truststore
Happens all the time. You import the CA into the keystore file and reference it as both. Kafka gets confused. Keep them separate.
2. Forgetting to Restart Brokers After Changing SSL Config
Kafka doesn’t hot-reload SSL keystores. You need a rolling restart. I use a script that checksums the keystore and triggers a restart only if changed.
3. Ignoring Certificate Expiry
Broker cert expires → clients can’t connect. Set up monitoring. Cert-manager for Kubernetes or a simple cron with openssl x509 -checkend works.
4. Not Validating Hostnames
Without ssl.endpoint.identification.algorithm=HTTPS, Kafka won’t check that the certificate’s CN/SAN matches the broker hostname. A MITM attack becomes trivial.
SSL in a Multi‑Broker Cluster
When you have 3+ brokers (kafka multi broker cluster setup), each needs its own certificate with its own hostname. But they all need to trust each other.
Set ssl.truststore.location to the same file on every broker. The keystore is unique per broker.
For inter‑broker communication, use the SSL listener as well:
properties
# On each broker
listener.security.protocol.map=INTERNAL:SSL,CLIENT:SSL,CONTROLLER:SSL
inter.broker.listener.name=INTERNAL
ssl.principal.mapping.rules=RULE:^CN=(.*?),.*/$1/
The ssl.principal.mapping.rules extracts the CN from the client certificate and uses it as the Kafka principal. This lets you set ACLs like “User:CN=kafka-broker-1” for inter‑broker authorization.
Performance Impact: What I’ve Measured
SSL adds CPU overhead. On modern hardware (2024+ Xeons with AES-NI), it’s about 5-8% throughput reduction for TLS 1.3. For Kafka’s wire protocol, that’s acceptable.
I benchmarked a 6-broker cluster at SIVARO with 128 partitions:
- Plaintext: 200K messages/sec per broker (1KB payloads)
- SSL (TLS 1.3, mutual auth): 185K messages/sec
Worth it. If you’re hitting CPU limits, use hardware security modules (HSMs) or QAT adapters. But 95% of teams don’t need that.
How SSL Fits into the Bigger Streaming Picture
You might be evaluating different streaming platforms. The security model varies. Pulsar, for example, has built-in SSL with per‑topic encryption keys. RabbitMQ uses AMQPS. But Kafka’s SSL model is dead simple — once you get past the keystore/truststore dance.
If you’re doing a deep comparison of Kafka vs Pulsar – Performance, Features, and Architecture, SSL is a table‑stakes feature for all of them. Where they differ is in certificate management at scale. Pulsar supports automatic cert rotation via its broker‑discovery service. Kafka requires manual or scripted rotation. That’s a pain point.
There’s also the debate about Pulsar vs Kafka – Comparison and Myths Explored. Some argue Pulsar’s geo‑replication with SSL is easier. I’ve run both. Kafka’s cert rotation is harder, but its tooling (MirrorMaker, Confluent Replicator) is more mature.
For teams stuck between Kafka vs Pulsar vs RabbitMQ vs NATS: What’s Actually Best for Your Use Case, I always say: pick the one with the simplest security model you can actually enforce. Kafka’s SSL is well‑documented and battle‑tested. That matters more than a feature checkbox.
But Wait — What About Kafka vs NiFi?
Sometimes people ask me about kafka vs nifi for data streaming. They’re different beasts. NiFi is an ETL tool with a UI. Kafka is a log‑based message bus. NiFi can use SSL to talk to Kafka — but you still need to secure Kafka at the broker level.
If you’re piping data from NiFi into Kafka, make sure the NiFi SSL properties match. I’ve seen mismatched cipher suites cause silent message drops.
Automating Certificate Renewal
Manual cert rotation doesn’t scale. For production, use:
- cert-manager on Kubernetes (with Let’s Encrypt or Vault)
- HashiCorp Vault PKI — issue short-lived certs (24 hours), auto-renew via agent
- ACME with custom CA
I lean toward Vault for Kafka because you can integrate its secrets engine directly with Kafka’s ssl.keystore.type=PEM (Kafka 3.4+). No JKS conversion.
Example with Vault agent:
hcl
# vault-agent.hcl
vault {
address = "https://vault.internal:8200"
ca_cert = "/etc/vault/ca.crt"
}
template {
contents = "{{ with secret "pki/issue/kafka" "common_name=kafka-broker-1.internal" "ttl=24h" }}{{ .Data.certificate }}{{ end }}"
destination = "/etc/kafka/ssl/broker-cert.pem"
command = "systemctl reload kafka"
}
Yes, you need a reload. But it’s faster than a full restart.
FAQ
Q: Can I use self‑signed certificates in production?
If you control all clients, yes. But you must distribute the CA cert to every client. If a client is compromised, they can’t trust anyone else. Self‑signed is fine for internal clusters. Don’t use it for internet‑facing Kafka.
Q: What’s the difference between SSL and TLS in Kafka?
Kafka uses “SSL” in config names, but actually negotiates TLS (preferably 1.3). Set ssl.enabled.protocols=TLSv1.2,TLSv1.3.
Q: Do I need mutual TLS (mTLS)?
If you need to authenticate clients, yes. If you only need encryption, set ssl.client.auth=none. But mTLS is better – no separate SASL mechanism.
Q: How do I migrate from plaintext to SSL without downtime?
Double the listeners: keep PLAINTEXT on a different port, add SSL. Move clients one by one to SSL, then remove PLAINTEXT. This works if your clients support multiple bootstrap servers.
Q: What about Kafka REST Proxy and Schema Registry?
They support SSL too. Configure them with the same truststore. Schema Registry can also enforce SSL for Avro serialization.
Q: Does SSL work with KRaft (no ZooKeeper)?
Yes. Kafka 3.8+ fully supports SSL for controller and metadata quorum. Configuration is identical.
Q: I get “javax.net.ssl.SSLHandshakeException: no cipher suites in common”. What gives?
Cipher suite mismatch. Check ssl.cipher.suites on both sides. Default is TLS_AES_256_GCM_SHA384. If you’re using older Java, add TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256.
Q: Is SSL enough? Or do I need SASL/SCRAM as well?
SSL authenticates machine identity. SASL authenticates user identity. For multi‑tenant clusters, use both: SSL for encryption, SASL/SCRAM or PLAIN for user auth. For single‑tenant, mTLS alone is fine.
Wrapping Up
Securing Kafka with SSL isn’t hard. But it’s easy to get wrong. I’ve seen teams skip it because “it’s just internal traffic.” Then a contractor’s laptop gets compromised, and suddenly your event stream is public.
Here’s the short version:
- Generate a CA.
- Create per‑broker certs with SANs.
- Enable SSL on all listeners.
- Set
ssl.client.auth=required. - Push client truststores everywhere.
- Monitor cert expiry.
Do this before you go to production. You can retrofit, but it’s ugly — and you’ll break something.
Now go encrypt your Kafka.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.