Support Questions
Find answers, ask questions, and share your expertise

Stream HISTORICAL tweets into HDFS

New Contributor

Dear Friends,


I have established a setup to stream tweets from Twitter into our HDFS baseed upon the keywords maintained in the FLUME configuration file.


I would like to know if it is possible for FLUME to load the historical/old tweets relevant to those keywords into the HDFS.
If this ain't possible, may I know whether we can achieve this in some other way?



Best Regards,



Master Guru
There's nothing in Flume that should prevent what you are looking to do. Flume sources feed off of whatever URL or other mechanism you provide it.

If Twitter provides a way for you to stream historical tweets based on a search keyword, provide the same to your Flume source and it will read what is presented to it.

Hi Ram,


are you able to pull older tweets in hdfs.