moving data from kafka to hdfs

Skodeto — Fri, 15 May 2020 12:12:20 GMT

I need a Help i have two question please

1-how I can transform the data from apache kafka to hdfs????

2-how I can transform the data from apache kafka to SparkStreming????

thank you

Re: moving data from kafka to hdfs

dyadav1 — Mon, 18 May 2020 07:31:22 GMT

You can use flume or nifi to publish data from kafka to nifi:

a. Using flume

Kafka Source -> Flume -> HDFS

b. Using Nifi:

Configure PublishKafka processor --> PutHdfs processor

And to integrate kafka for spark streaming you need to build spark streaming job, refer the below doc. for more details: