Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

how to pull tweets and store it in kafka ?

Expert Contributor

hello,

I want to fetch tweets and store into the kafka. how it is possible. ?

I am using hortonworks on single machine.

1 REPLY 1

@heta desai

One option is to use Nifi/HDF to get the tweets from twitter and then post them to Kafka. Take a look at the demo in the link below to give you an idea ho that might work. The destination will be Kafka though rather than HDFS or Solr (which is what the example uses).

https://community.hortonworks.com/articles/1282/sample-hdfnifi-flow-to-push-tweets-into-solrbanana.h...

Alternatively, you can write a Java producer using the Twitter API to get the tweets and put them in Kafka as per the examples below.

https://acadgild.com/blog/streaming-twitter-data-using-kafka/

https://www.tutorialspoint.com/apache_kafka/apache_kafka_real_time_application.htm