i'am trying to develop my first project with horthonworks so i need to stream data from (not twitter) facebook ,web site ,instagram , using spring-xd , apache flume,or nifi .Does anyone have any examples/templates for using this plugins .

[#1 · Accepted solution]

@nejm hadj

Nifi and Facebook https://community.hortonworks.com/questions/8851/nifi-gethttp-processor-for-facebook-api-error.html

[#2]

@nejm hadj Also, see this https://community.hortonworks.com/articles/961/a-collection-of-nifi-examples.html

Search in HCC on Nifi examples

[#3]

@nejm hadj Adding more information based on your comments

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.flume.ExecuteFlumeSink/...

You should stick with NiFi and use built in processor to ingest the data from various social media sources.

Please do read docs.

In NiFi, the contents of a FlowFile are accessed via a stream, but in Flume it is stored in a byte array. This means the full content will be loaded into memory when a FlowFile is processed by the ExecuteFlumeSink processor. You should consider the typical size of the FlowFiles you'll process and the batch size, if any, your sink is configured with when setting NiFi's heap size.

[#4]

@nejm hadj

nifi templates Link

[#5]

thnx , i'am confused about what i will use for streaming data ( nifi or flume)to hdfs ,what is the difference between this plugins ,my goal is streaming data from social media (facebook ,instagram,website,mobile) . i can use them together ??

[#6]

@nejm hadj nifi is a visual tool, flume is not. You can work with nifi and flume together as nifi has flume processors. Once you try nifi you won't need to look at flume.