一个spark receiver 或多个spark receiver 接收 多个flume agent

利用一个spark receiver 接收多个flume agentString host = args[0]; int port = Integer.parseInt(args[1]); String host1 = args[2]; int port1 = Integer.parseInt(args[3]); InetSocketAddress address1 = new InetSocketAddress(host,port); InetSocketAddress address2 = new InetSocketAddress(host1,port1); InetSocketAddress[] InetSocketAddressArray = {address1,address2};JavaStreamingContext jssc = new JavaStreamingContext(new SparkConf().setAppName(“JavaFlumeEventHandle_1”), Durations.seconds(2)); JavaReceiverInputDStream<SparkFlumeEvent> flumeStream = FlumeUtils.createPollingStream(jssc, InetSocketAddressArray, StorageLevel.MEMORY_AND_DISK_SER_2());2个spark receiver接收多个flume agentString host = args[0];int port = Integer.parseInt(args[1]);String host1 = args[2];int port1 = Integer.parseInt(args[3]);InetSocketAddress address1 = new InetSocketAddress(host,port);InetSocketAddress address2 = new InetSocketAddress(host1,port1);InetSocketAddress[] InetSocketAddressArray = {address1,address2};JavaStreamingContext jssc = new JavaStreamingContext(new SparkConf().setAppName(“JavaFlumeEventHandle_1”), Durations.seconds(2));//JavaReceiverInputDStream<SparkFlumeEvent> flumeStream = FlumeUtils.createPollingStream(jssc, InetSocketAddressArray, StorageLevel.MEMORY_AND_DISK_SER_2());JavaReceiverInputDStream<SparkFlumeEvent> flumeStream = FlumeUtils.createPollingStream(jssc,host,port);JavaReceiverInputDStream<SparkFlumeEvent> flumeStream1 = FlumeUtils.createPollingStream(jssc, host1, port1);JavaDStream<SparkFlumeEvent> union = flumeStream.union(flumeStream1);

性能参照:

,与其在那里苦苦挣扎,碍于面子硬撑,倒不如微笑着面对,

一个spark receiver 或多个spark receiver 接收 多个flume agent

相关文章:

你感兴趣的文章:

标签云: