KRaft mode Kafka on Kubernetes

Resources to accompany the article Designing and testing a highly available Kafka cluster on Kubernetes on Learnk8s. The article aims to distill and clearly explain the essential considerations for high availability when combining Kafka and Kubernetes.

KRaft mode Kafka on Kubernetes is the simplest example of a Kafka cluster on Kubernetes. This repo is forked from that.

Acknowledgements

Josh Purcell for creating the original source repository and accompanying tutorial for this project is available here on IBM Developer (and possibly cross-posted on other sites).

Changes

All changes are in the commit log. To summarise:

• Bump kafka to version 3.7.0.

• Externalise CLUSTER_ID env. Set default replication properties.
• Externalise ADD_LISTENERS , ADD_ADVERTISED_LISTENERS, ADD_LISTENER_SECURITY_PROTOCOL_MAP.
• Configure statefulset for improved availability.

• Introduced podDisruptionBudget.
• JRE slim image

Contents

Instructions for how to use this repo are found in the tutorial (links above).

• docker: kafka Dockerfile and entrypoint shell script
• kubernetes: kubernetes cluster config files managed by minikube

Testing

To test the container alone, the container requires a CLUSTER_ID environment variable and hostname of the format "kafka-n" (where n is an integer).

Here's an example launch from docker.

$ docker run --rm -e CLUSTER_ID=oh-sxaDRTcyAr6pFRbXyzA --hostname kafka-0 tode/kafka-kraft:3.7.0 kafka-0