Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to stream data from live api to HDFS

Highlighted

how to stream data from live api to HDFS

Expert Contributor

How can I stream data from google analytics realtime API to HDFS.We have so far been using R and putting the files in HDFS for bulk imports but would like to know best and possible approaches for GA realtime API.

3 REPLIES 3

Re: how to stream data from live api to HDFS

Rising Star

Use nifi to make the API call to google analytics and push the data into HDFS preferably close to the HDFS block size to avoid small file problem.

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.1.1/bk_user-guide/content/introduction.html

Highlighted

Re: how to stream data from live api to HDFS

Expert Contributor

Sure. But could you please explain how to use it with google analytics API? How do I stream data from a pull API?Also, how would the authentication work? I have used nifi once, liked it and would love to use it again. It's an amazing tool but very limited support for nifi makes it rather convenient to pick other tools over it.

Highlighted

Re: how to stream data from live api to HDFS

Expert Contributor
Don't have an account?
Coming from Hortonworks? Activate your account here