Trying to make multiple request to Facebook Graph API with Nifi

[#1]

Hi,

I need to make a lot of request to the Facebook Graph API using Nifi.

I looked at the documentation https://developers.facebook.com/docs/graph-api/making-multiple-requests which tells me that i need to POST a JSON which contains the request i want to make in a batch format to a specific endpoint.

So i created a Nifi template with 3 processors : GetFile -->PostHTTP-->PutHDFS.

GetFile reads a very simple JSON file i made to test.

When i run the template, i have no error message. However, the HDFS file wrote is just a copy of my initial JSON file.

Any explanation / solution?

Thank you in advance.

[#2 · Accepted solution]

I don't think PostHTTP outputs the response as a flow file. If you are looking for the response to be in a flow file, try InvokeHttp (with HTTP Method property set to "POST") instead and send its "response" relationship to your PutHDFS processor.

[#3]

@mburgess Thank you for your answer. InvokeHttp seems to work since i got an answer from the graph API that i can write in HDFS.

However, i didn't manage to make this answer depend on my original JSON : whatever is writen in the file taken by the GetFile processor, even if it is not a JSON formated file, the answer is the same. I looked at the nifi documentation on InvokeHttp and tried to put something in the property "Attributes to send" but it did nothing so far.

Do you know how InvokeHttp interacts with the FlowFiles coming from GetFile?

[#4]

@Lubin Lemarchand

Since the information is splitted across two threads, I am putting it here for future references. This is how you can do batch requests on Facebook Graph API with InvokeHTTP:

• GetFile processor: the input file will contain the requests that you want to send to Facebook (in JSON) as well as your access_token
  • Input directory: /tmp/input
• InvokeHTTP processor: passes the requests to the Graph API. Use these configurations:
  • HTTP Method: POST
  • URL: https://graph.facebook.com
• PutFile processor: Save the response JSON in a file
  • Directory: /tmp/output

Here is an example of an input file to execute two requests, get my Facebook url and get my friends list

batch=[{"method":"GET", "relative_url":"me"},{"method":"GET", "relative_url":"me/friends"}]&access_token=your-access-token

[#5]

Thank you very much. It works splendidly. I'm not too sure of what was wrong in my json but it doesn't matter anymore.

[#6]

Hi I am working on Small project for my school to get data from Facebook and do some analysis. If you have worked to get data from facebook using nifi, can you share your dashboard?

[#7]

@Abdelkrim Hadjidj

@Lubin Lemarchand

Can you please let me know in Unix How can I write values in file for GetFile Processor because symbol "∾" changed into some junk value when I write it in unix flat file.