Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to pull data from API and store it in HDFS

avatar
Expert Contributor

I am aware of flume and Kafka but these are event driven tools. I don't need it to be event driven or real time but may be just schedule the import once in a day.

What are the data ingestion tools available for importing data from API's in HDFS?

I am not using HBase either but Hive. I have used `R` language for that for quite a time but I am looking for a more robust,may be native solution to Hadoop environment.

1 ACCEPTED SOLUTION

avatar
Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
2 REPLIES 2

avatar
Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Master Guru

NIFI/HDF is the way, very easy and a huge number of sources.

https://community.hortonworks.com/articles/52415/processing-social-media-feeds-in-stream-with-apach....

ttps://community.hortonworks.com/content/kbentry/47854/accessing-facebook-page-data-from-apache-nifi.html h

ttps://community.hortonworks.com/articles/46258/iot-example-in-apache-nifi-consuming-and-producing.html h

ttps://community.hortonworks.com/articles/45531/using-apache-nifi-070s-new-putslack-processor.html h

ttps://community.hortonworks.com/articles/45706/using-the-new-hiveql-processors-in-apache-nifi-070.html h

https://community.hortonworks.com/content/kbentry/44018/create-kafka-topic-and-use-from-apache-nifi-...

https://community.hortonworks.com/content/kbentry/55839/reading-sensor-data-from-remote-sensors-on-r...