Example of Apache Kafka + Spark Streaming + HBase Production Real Time data

[#1]

hello,

I Want to make a demo which includes kafka, spark and Hbase. For example having files store it in kafka topic, then spark streaming will read and navigate the files from kafka topic and performs some filtering analysis and stores the records into the Hbase. before saving it hbase, first I need to check the resulted record in Hbase whether if it exists or not based on its' date.

also I want to ask something else, can I join table from hive with the data files before performing spark analysis from spark.

can anyone please provide a step by step guide in java. I am new to hadoop.

Thank you

[#2]

@Noor elkindy Perhaps you can use this demo as reference:

https://github.com/james94/iot-truck-streaming

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.