kafka producer 发送消息简介
kafka 的 topic 由 partition 组成，producer 会根据 key，选择一个 partition 发送消息，而 partition 有多个副本，副本有 leader 和 follower 之分，producer 需要把消息发送到 partition 的 leader。
背景：kafka 集群由 3 个节点组成(node1，node2，node3，其中 node3 是 controller)，存在名为 idea-zhang 的 topic，只有一个 partition， partition 的副本数是2。
流程图如下：
重要日志如下：
Initialize connection to node 10.253.114.163:9092 (id: -1 rack: null) for sending metadata request (org.apache.kafka.clients.NetworkClient)
Sending metadata request (type=MetadataRequest, topics=idea-zhang) to node 10.253.114.163:9092 (id: -1 rack: null) (org.apache.kafka.clients.NetworkClient)
Completed receive from node -1 for METADATA with correlation id 1, received
{
throttle_time_ms = 0,
brokers = [{
node_id = 2,
host = 10.253.162.17,
port = 9092,
rack = null
},
{
node_id = 1,
host = 10.253.160.154,
port = 9092,
rack = null
},
{
node_id = 3,
host = 10.253.114.163,
port = 9092,
rack = null
}],
cluster_id = oX31KEPESLeIWHz7BgJv7g,
controller_id = 3,
topic_metadata = [{
error_code = 0,
topic = idea-zhang,
is_internal = false,
partition_metadata = [{
error_code = 0,
partition = 0,
leader = 2,
replicas = [2, 3],
isr = [2, 3]
}]
}]
}
(org.apache.kafka.clients.NetworkClient)
Initiating connection to node 10.253.162.17:9092 (id: 2 rack: null) (org.apache.kafka.clients.NetworkClient)
Sending record ProducerRecord(topic=idea-zhang, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=1, value=Message_1, timestamp=null) with callback null to topic idea-zhang partition 0 (org.apache.kafka.clients.producer.KafkaProducer)
注意！
本站转载的文章为个人学习借鉴使用，本站对版权不负任何法律责任。如果侵犯了您的隐私权益，请联系我们删除。