How to get facebook feeds into NiFi?

[#1]

How to get facebook data and store into my HDFS

• Ex. I have one #tag say Starbucks i need to get feeds from it as json file
• I'm newer in NIFI so please explain from beginning and read many posts regarding facebook need to start from getHttp
• which address i need to give to get feeds for particular tags.
• please show me properties with example

Thanks in advance

[#2]

Hi @Manikandan Durairaj,

I'd suggest you to read the following posts:

• https://developers.facebook.com/docs/graph-api/using-graph-api
• https://developers.facebook.com/docs/graph-api/common-scenarios
• https://developers.facebook.com/docs/facebook-login/access-tokens

I'd try something like:

https://graph.facebook.com/search?q=%23YOU-HASH-VALUE&limit=10000&access_token=YOUR-ACCESS-TOKEN

Hope this helps.

[#3]

Thanks @Pierre Villard

I got some data but still having doubts

• I have created one app in facebook developer and using graph API explorer get my access token
• In my Gethttp, I use url as https://graph.facebook.com/v2.6/Starbucks&access_token=my_access_token
• i have to give at least a name of the file and i configure ssl (StandardSSLContextService)
• Finally i directly put into HDFS, after receive first file it shows error file already exist

My problem is how can i received as multiple json files and my access token is expired in specific time is thr any options to extend dynamically....please show me with properties screen shot

Thank you...

[#4]

Hi @Manikandan Durairaj

To update the name of the file, you can use an UpdateAttribute processor and set the attribute 'filename' with the name of your choice using expression language [1]. If you want a name based on current time, you can use date functions, otherwise you can use the function UUID to generate a unique filename for each flow file.

That would be a property filename with ${UUID()}

[1] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html

Hope this helps.

[#5]

This post is similar to your use case and shows how to rename the file for repeated ingests as @Pierre Villard describes

https://community.hortonworks.com/articles/48816/nifi-to-ingest-and-transform-rss-feeds-to-hdfs-usi....

[#6]

Hi @Manikandan Durairaj,

Can you share template with me if you get success on this.

Regards,

Yogesh

[#7]

Did you ever run into problems in terms of API permission? I am trying to do something similar, searching for a specific word in public posts and fetch them using NiFi and pushing it to HDFS but I am having trouble doing so.

Could you share a template with me if you get any further in this process?

Thanks

[#8 · Accepted solution]

I wrote this for basic facebook ingest

https://community.cloudera.com/t5/Community-Articles/Accessing-Facebook-Page-Data-from-Apache-NiFi-1...

[#9]

The link is no longer valid. Could you please update?

Thanks in advance.

[#10]

@Mostafa01 The link in Tim's reply above has been updated and should correctly resolve now.