Created 01-09-2016 11:05 PM
Actually iam trying to use any api so that i can stream data using nifi and put it to hdfs. I know GetHttp processor is used to fetch data from a website. I want to know any free api thats streams data and also which processor to use in Nifi to connect to that api and stream data from that api using nifi to hdfs. Its a prototype iam trying to build using nifi. I just want to use an api other than twitter. Appreciate any help from friends.
Created 01-11-2016 02:03 PM
I am coming in a bit late but want to also recommend randomuser.me
Also want to point out that there are plenty of nifi templates found here: https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates
Look at the "Fun_with_Hbase" template which gets random data from a rest API and in your case use the PutHDFS processor to ingest in to HDFS.
Created 01-10-2016 11:04 PM
This shows how to use Google Finance API: https://community.hortonworks.com/content/kbentry/8422/visualize-near-real-time-stock-price-changes-...
Created 01-11-2016 09:23 AM
Thanks Zlatkin
Created 01-11-2016 02:03 PM
I am coming in a bit late but want to also recommend randomuser.me
Also want to point out that there are plenty of nifi templates found here: https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates
Look at the "Fun_with_Hbase" template which gets random data from a rest API and in your case use the PutHDFS processor to ingest in to HDFS.
Created 01-11-2016 05:27 PM
Thanks very much guys for the response. Iam planning to use datasift streaming api and rest api using nifi to store data in hdfs or hive or hbase tables? So i would appreciate your help in choosing the right approach and right nifi processors. Shall i go ahead and use GetHttp and PutHdfs here for both streaming and rest api or shall i think about streaming data directly from nifi to hbase or hive?? please let me know @Juan Sequeiros, @Vladimir Zlatkin thanks guys