Parameters used:
| Parameter | Comment |
| zookeeper_host | the list of Zookeeper nodes in cluster |
| topic_name | the topic name |
| broker_list | the list of Kafka brokers in cluster |
| replication_factor | the replication factor for topic |
| partition_count | the number of partitions for topic |
Topic management:
- create a topic:
bin/kafka-topics.sh --create --zookeeper [zookeeper_list] --replication-factor [replication_factor] --partitions [partition_count] --topic [topic_name] Example: >bin/kafka-topics.sh --create --zookeeper zookeeeper_nd1:2181,zookeeper_nd2:2181,zookeeper_nd3:2181 --replication-factor 1 --partitions 3 --topic TestKafkaTopicName
- view list of topics
bin/kafka-topics.sh --list --zookeeper [zookeeper_list] Example: >bin/kafka-topics.sh --list --zookeeper zookeeeper_nd1:2181,zookeeper_nd2:2181,zookeeper_nd3:2181
- describe a topic
bin/kafka-topics.sh --describe --zookeeper [zookeeper_list] --topic [topic_name] Example: >bin/kafka-topics.sh --describe --zookeeper zookeeeper_nd1:2181,zookeeper_nd2:2181,zookeeper_nd3:2181 --topic TestKafkaTopicName
Messaging:
- produce a message to Kafka
bin/kafka-console-producer.sh --broker-list [broker_list] --topic [topic_name] Example: >bin/kafka-console-producer.sh --broker-list kafka_nd1:6667,kafka_nd2:6667,kafka_nd3:6667 --topic TestKafkaTopicName Note: you have to be extremely accurate when sending a message to Kafka from console because the message might be in incorrect format that would break the consumer logic.
- consume all messages from Kafka topic (old ones and new arriving)
bin/kafka-console-consumer.sh --zookeeper [zookeeper_list] --topic [topic_name] --from-beginning Example: >bin/kafka-console-consumer.sh --zookeeper zookeeeper_nd1:2181,zookeeper_nd2:2181,zookeeper_nd3:2181 --topic TestKafkaTopicName
- consume the messages arriving in Kafka topic
bin/kafka-console-consumer.sh --zookeeper [zookeeper_list] --topic [topic_name] --from-beginning Example: >bin/kafka-console-consumer.sh --zookeeper zookeeeper_nd1:2181,zookeeper_nd2:2181,zookeeper_nd3:2181 --topic TestKafkaTopicName
Software Engineering Blog 