Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to stream data from a webhook into hdfs or hive

How to stream data from a webhook into hdfs or hive

Expert Contributor

How to get data from a webhook and put that in hdfs in real-time? If not real-time ,what are the tools available? I do not want touse a web server to consume this data and then put it in hdfs.

I guess nifi can do something like that. Do we have an example of nifi consuming data from a webhook? Any help or direction is appreciated.

3 REPLIES 3
Highlighted

Re: How to stream data from a webhook into hdfs or hive

Expert Contributor

Sounds like you want to use the GetHttp processor.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.2.0/org.apache...

There is an example on the NiFi wiki,

https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates

and I'm sure there are a few in the Articles and Repos sections on HCC.

Have a great day!

Re: How to stream data from a webhook into hdfs or hive

Expert Contributor

But I don't see an example for streaming from webhook. I am new to it and would like a guide for this.

Re: How to stream data from a webhook into hdfs or hive

New Contributor

I have the same need. I would like to use Nifi in a reactive way. Instead of sending getHTTP process time to time, I would like to be able to subscribe to endpoints and listen events that come from them. Thats possible?

Thanks in advance