I have setup a kafka cluster using strimzi on minikube. And the minikube running on an ec2 machine. I have created a nodeport type service to expose the kafka cluster. But I can not access the kafka cluster outside of the ec2.

I used kubectl port-forward to access the kafka cluster on ec2 ip-address. But could not connect to the broker.

3

1 Answer

You cannot easily use kubectl port-forward because of how the Kafka protocol works and how clients need to connect directly to each of the brokers (depending on what partition replicas it hosts etc.). You would need to:

  • Port-forward all the brokers
  • Configure the advertised hostnames/ports for them in the Kafka custom resource to your localhost and the port you exposed it on for each broker

Using the type: nodeport listener might be easier. But a lot depends on how your Kubernetes cluster is configured and how you run Kubernetes. To make it work, you would need to make sure that:

  • The node port is exposed on the EC2 host and not just on some VM or container inside it
  • Your Kubernetes node is configured with the right node addresses (e.g. the public IP of the EC2 host). You can check that using kubectl get node -o yaml command. The advertised host in the broker will be based on this address.

Strimzi lets you customize all of these if needed. But without knowing the exact environment, it is not easy to provide the exact YAML. So you would need to try it. This blog post series might help you understand how Kafka does this and how to configure it in Strimzi:

2

Your Answer

Sign up or log in

Sign up using Google Sign up using Facebook Sign up using Email and Password

Post as a guest

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.