最近升级调试一个storm任务，出了一个很奇怪的internal exception，堆栈如下：
backtype.storm.daemon.worker – Received invalid messages for unknown tasks. Dropping…backtype.storm.daemon.worker – Received invalid messages for unknown tasks. Dropping…backtype.storm.daemon.worker – Received invalid messages for unknown tasks. Dropping…2015-01-28 18:25:52 [Thread-11-worker-receiver-thread-0] WARN backtype.storm.daemon.worker – Received invalid messages for unknown tasks. Dropping…2015-01-28 18:29:12 [Thread-12-disruptor-worker-transfer-queue] ERROR backtype.storm.util – Async loop died!java.lang.RuntimeException: java.lang.RuntimeException:Client is being closed, and does not take requests any more at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:127) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:96) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:81) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.disruptor$consume_loop_STAR_$fn__1681.invoke(disruptor.clj:95) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.util$async_loop$fn__457.invoke(util.clj:457) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] at java.lang.Thread.run(Thread.java:662) [na:1.6.0_38]Caused by: java.lang.RuntimeException: Client is being closed, and does not take requests any more at backtype.storm.messaging.netty.Client.send(Client.java:183) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.utils.TransferDrainer.send(TransferDrainer.java:54) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__4610$fn__4611.invoke(worker.clj:330) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__4610.invoke(worker.clj:328) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.disruptor$clojure_handler$reify__1668.onEvent(disruptor.clj:59) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:124) ~[storm-core-0.9.3-rc1.jar:0.9.3-rc1] … 6 common frames omitted2015-01-28 18:29:12 [Thread-12-disruptor-worker-transfer-queue] ERROR backtype.storm.util – Halting process: ("Async loop died!")java.lang.RuntimeException: ("Async loop died!") at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:319) [storm-core-0.9.3-rc1.jar:0.9.3-rc1] at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na] at backtype.storm.disruptor$consume_loop_STAR_$fn__1679.invoke(disruptor.clj:93) [storm-core-0.9.3-rc1.jar:0.9.3-rc1] at backtype.storm.util$async_loop$fn__457.invoke(util.clj:467) [storm-core-0.9.3-rc1.jar:0.9.3-rc1] at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] at java.lang.Thread.run(Thread.java:662) [na:1.6.0_38]
运行了一段时间以后（大概一个小时），就会不停地抛出Received invalid messages for unknown tasks. Dropping… 这样的warning，之后便是async loop died,worker会不间断地重启
对应supervisor日志：
2015-02-02 17:46:51 b.s.d.supervisor [INFO] Shutting down and clearing state for id eec730d3-7487-457d-b22c-29a3691d7236. Current supervisor time: 1422870408. State::timed-out, Heartbeat: #backtype.storm.daemon.common.WorkerHeartbeat{:time-secs 1422870376, :storm-id "xxxxtopology-TopologyUAT-412-1422861787", :executors #{[3 3] [136 136] [110 110] [83 83] [55 55] [29 29] [-1 -1]}, :port 6711}
2015-02-02 17:22:14 b.s.d.supervisor [INFO] Shutting down and clearing state for id d940a373-37d8-4dee-b206-0a42be3d22da. Current supervisor time: 1422868934. State: :disallowed, Heartbeat: #backtype.storm.daemon.common.WorkerHeartbeat{:time-secs 1422868889, :storm-id "xxxtopology-TopologyUAT-412-1422861787", :executors #{[35 35] [144 144] [18 18] [117 117] [89 89] [63 63] [-1 -1]}, :port 6717}
从日志上只能拿到以上的信息，不管是worker、supervisor的日志都看起来原因不详，一时之间无法定位问题。
1.Netty的接收线程无法正常获取，，为什么？
大多数人想要改造这个世界，但却罕有人想改造自己。
