kafka

http://kafka.apache.org/07/quickstart.html

https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer

Consider using Samza which is an abstraction over KAfka!
http://samza.apache.org/learn/documentation/latest/introduction/background.html