- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Using Nifi to stream data from Free streaming(not twitter) api to hdfs
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
Created ‎01-09-2016 11:05 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Actually iam trying to use any api so that i can stream data using nifi and put it to hdfs. I know GetHttp processor is used to fetch data from a website. I want to know any free api thats streams data and also which processor to use in Nifi to connect to that api and stream data from that api using nifi to hdfs. Its a prototype iam trying to build using nifi. I just want to use an api other than twitter. Appreciate any help from friends.
Created ‎01-11-2016 02:03 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am coming in a bit late but want to also recommend randomuser.me
Also want to point out that there are plenty of nifi templates found here: https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates
Look at the "Fun_with_Hbase" template which gets random data from a rest API and in your case use the PutHDFS processor to ingest in to HDFS.
Created ‎01-10-2016 11:04 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This shows how to use Google Finance API: https://community.hortonworks.com/content/kbentry/8422/visualize-near-real-time-stock-price-changes-...
Created ‎01-11-2016 09:23 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Zlatkin
Created ‎01-11-2016 02:03 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am coming in a bit late but want to also recommend randomuser.me
Also want to point out that there are plenty of nifi templates found here: https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates
Look at the "Fun_with_Hbase" template which gets random data from a rest API and in your case use the PutHDFS processor to ingest in to HDFS.
Created ‎01-11-2016 05:27 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks very much guys for the response. Iam planning to use datasift streaming api and rest api using nifi to store data in hdfs or hive or hbase tables? So i would appreciate your help in choosing the right approach and right nifi processors. Shall i go ahead and use GetHttp and PutHdfs here for both streaming and rest api or shall i think about streaming data directly from nifi to hbase or hive?? please let me know @Juan Sequeiros, @Vladimir Zlatkin thanks guys
