Support Questions
Find answers, ask questions, and share your expertise

how to pull tweets and store it in kafka ?

how to pull tweets and store it in kafka ?

Expert Contributor

hello,

I want to fetch tweets and store into the kafka. how it is possible. ?

I am using hortonworks on single machine.

1 REPLY 1
Highlighted

Re: how to pull tweets and store it in kafka ?

@heta desai

One option is to use Nifi/HDF to get the tweets from twitter and then post them to Kafka. Take a look at the demo in the link below to give you an idea ho that might work. The destination will be Kafka though rather than HDFS or Solr (which is what the example uses).

https://community.hortonworks.com/articles/1282/sample-hdfnifi-flow-to-push-tweets-into-solrbanana.h...

Alternatively, you can write a Java producer using the Twitter API to get the tweets and put them in Kafka as per the examples below.

https://acadgild.com/blog/streaming-twitter-data-using-kafka/

https://www.tutorialspoint.com/apache_kafka/apache_kafka_real_time_application.htm