A Sivaro book · 13 chapters
Kafka and event streaming
From late reports and systems that disagree, to what event streaming is, who needs it, what it takes, and how to run Kafka well.
Chapters
- 01 What is Kafka Connect Used For? A Practitioner's GuideWhat Kafka Connect is used for: moving data between Kafka and other systems without custom code. Covers CDC, cloud storage sinks, alternatives, tuning.
- 02 How to Build a Kafka Producer in PythonBuild a reliable Kafka producer in Python: confluent-kafka or kafka-python, serialization, batching, idempotence, error handling, monitoring, security.
- 03 Kafka Producer Configuration Best Practices: A 2026 GuideKafka producer settings worth tuning: acks, compression, batching, buffer memory, retries, partitioning and security, plus monitoring and common traps.
- 04 Kafka Topic Partition Strategy Best Practices for 2026Kafka partition strategy: why partition count matters, key cardinality traps, throughput versus ordering, right-sizing, rebalancing and lag monitoring.
- 05 Kafka Schema Registry Setup GuideHow to set up Confluent Schema Registry for Kafka: install, register Avro schemas, set up producers and consumers, compatibility rules, pitfalls.
- 06 Kafka Security: SASL vs OAuth AuthenticationSASL or OAuth for Kafka authentication? Compares SASL/SCRAM with OAuth, token validation, rebalancing effects, and a step-by-step OAuth rollout plan.
- 07 How to Scale Kafka Consumers Without Breaking ProductionHow to scale Kafka consumers safely: partitions cap parallelism, rebalance triggers, eager versus cooperative protocols, timeout tuning and lag monitoring.
- 08 Kafka Best Offset Management Strategies – A No-BS GuideKafka offset management: auto versus manual commit, consumer groups, lag monitoring, idempotent consumers, rebalances and external offset storage.
- 09 How to Delete Kafka Topic and Reset Offsets: Practical GuideHow to delete a Kafka topic and reset consumer offsets safely: broker config, why deletes fail, dry runs, resets by timestamp, rebalancing and lag checks.
- 10 Kafka vs Pulsar Use Cases (2026)Kafka or Pulsar? Compares architecture, stream processing, multi-tenancy, geo-replication and scaling, covers RabbitMQ as an option, and how to choose.
- 11 Kafka vs Redpanda Performance: A Practical GuideKafka versus Redpanda performance: consumer group rebalancing, latency jitter, throughput at scale, ZooKeeper costs, and when Kafka is the better choice.
- 12 Kafka vs NiFi for Data Streaming: A Practitioner's Guide (2026)Kafka or NiFi for data streaming? Compares stream processing with data flow orchestration, performance, operations and cost, and when to use both.
- 13 Temporal Workflow Engine Comparison: What Actually Works in ProductionTemporal compared with Cadence, Step Functions, Dapr, Conductor, Zeebe, Airflow and Kestra. Covers durable execution, job queues and how to choose.