下载了最新的版本0.8.0 Beta1 Release
下载地址：https://dist.apache.org/repos/dist/release/kafka/kafka-0.8.0-beta1-src.tgz
一、安装
# tar xzvf kafka-0.8.0-beta1-src.tgz# cd kafka-0.8.0-beta1-src# ./sbt update# ./sbt package# ./sbt assembly-package-dependency
./sbt update这一步有点慢，因为需要下载很多依赖文件
安装ok
二、启动服务
首先启动zookeeper服务，使用kafka提供的单节点脚本
# bin/zookeeper-server-start.sh config/zookeeper.properties &
“&”号是为了让在后台运行，要不还要在手动放后台或者重新开启一个终端窗口。
然后启动kafka服务，同样有现成的脚本
# bin/kafka-server-start.sh config/server.properties &
同样放后台
三、创建topic
创建一个topic，，名字叫test
# bin/kafka-create-topic.sh –zookeeper localhost:2181 –replica 1 –partition 1 –topic test
$bin/kafka-create-topic.shMissing required argument OptionDescription——————-partition <Integer: # of partitions> number of partitions in the topic(default: 1)–replica <Integer: replication factor> replication factor for each partitionsin the topic (default: 1)–replica-assignment-listfor manually assigning replicas to<broker_id_for_part1_replica1 :brokers (default: )broker_id_for_part1_replica2,broker_id_for_part2_replica1 :broker_id_for_part2_replica2, …>–topic <topic>REQUIRED: The topic to be created.–zookeeper <urls>REQUIRED: The connection string forthe zookeeper connection in the formhost:port. Multiple URLS can begiven to allow fail-over.
查看创建的topic
# bin/kafka-list-topic.sh –zookeeper localhost:2181 2>/dev/nulltopic: test partition: 0 leader: 0 replicas: 0 isr: 0
“2>/dev/null”屏蔽一些线程的启动信息等,只显示想看到的结果。
四、收发消息
kafka发送的message可以是文件或者标准输入的一行。
#bin/kafka-console-producer.sh –broker-list localhost:9092 –topic test >/dev/nullhahahahahaha
收取message
# bin/kafka-console-consumer.sh –zookeeper localhost:2181 –topic test –from-beginning 2>/dev/nullhahahahahaha
五、多broker测试
之前是一台机器上当做一个node，现在尝试在一台机器上放3个node，即broker
修改配置文件
# cp config/server.properties config/server-1.properties# cp config/server.properties config/server-2.propertiesconfig/server-1.properties:broker.id=1port=9093log.dir=/tmp/kafka-logs-1 config/server-2.properties:broker.id=2port=9094log.dir=/tmp/kafka-logs-2
启动broker1和broker2
# JMX_PORT=9997 bin/kafka-server-start.sh config/server-1.properties &# JMX_PORT=9998 bin/kafka-server-start.sh config/server-2.properties &
需要设置JMX的端口，防止冲突
可以jps看一下任务
# jps23398 Kafka23300 QuorumPeerMain24057 Kafka23944 Kafka28938 Jps
创建一个topic
# bin/kafka-create-topic.sh –zookeeper localhost:2181 –replica 3 –partition 1 –topic my-replicated-topic
list一下
# bin/kafka-list-topic.sh –zookeeper localhost:2181topic: my-replicated-topic partition: 0 leader: 2 replicas: 2,0,1 isr: 2,0,1topic: test partition: 0 leader: 0 replicas: 0 isr: 0
partition：同一个topic下可以设置多个partition，将topic下的message存储到不同的partition下，目的是为了提高并行性
leader：负责此partition的读写操作，每个broker都有可能成为某partition的leader
replicas：副本，即此partition在那几个broker上有备份，不管broker是否存活
isr：存活的replicas
测试：删除一个broker之后
心有多大，舞台就有多大
