question i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins . in Support Questions

i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins .

nejmhadjmbarek — Thu, 04 Feb 2016 19:18:56 GMT

Re: i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins .

nsabharwal — Thu, 04 Feb 2016 19:21:15 GMT

@nejm hadj

Nifi and Facebook https://community.hortonworks.com/questions/8851/nifi-gethttp-processor-for-facebook-api-error.html

Re: i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins .

aervits — Thu, 04 Feb 2016 19:23:03 GMT

@nejm hadj

nifi templates Link

Re: i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins .

nsabharwal — Thu, 04 Feb 2016 19:23:18 GMT

@nejm hadj Also, see this https://community.hortonworks.com/articles/961/a-collection-of-nifi-examples.html

Search in HCC on Nifi examples

Re: i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins .

nejmhadjmbarek — Fri, 05 Feb 2016 17:53:13 GMT

thnx , i'am confused about what i will use for streaming data ( nifi or flume)to hdfs ,what is the difference between this plugins ,my goal is streaming data from social media (facebook ,instagram,website,mobile) . i can use them together ??

Re: i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins .

aervits — Fri, 05 Feb 2016 19:14:58 GMT

@nejm hadj nifi is a visual tool, flume is not. You can work with nifi and flume together as nifi has flume processors. Once you try nifi you won't need to look at flume.

Re: i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins .

nsabharwal — Mon, 29 Feb 2016 19:02:59 GMT

@nejm hadj Adding more information based on your comments

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html

You should stick with NiFi and use built in processor to ingest the data from various social media sources.

Please do read docs.

In NiFi, the contents of a FlowFile are accessed via a stream, but in Flume it is stored in a byte array. This means the full content will be loaded into memory when a FlowFile is processed by the ExecuteFlumeSink processor. You should consider the typical size of the FlowFiles you'll process and the batch size, if any, your sink is configured with when setting NiFi's heap size.