How to send headers to InvokeHTTP process.?

[#1]

Hi,

i am trying to download an object from google cloud storage using OAuth API. i got the access token.

i can see the objects metadata when i use the Remote URL in InvokeHTTP as as

https://www.googleapis.com/storage/v1/b/tarapareddy/o/test2?access_token={access_token}

{
 "kind": "storage#object",
 "id": "tarapareddy/test2/1522956104046841",
 "selfLink": "https://www.googleapis.com/storage/v1/b/tarapareddy/o/test2",
 "name": "test2",
 "bucket": "tarapareddy",
 "generation": "1522956104046841",
 "metageneration": "1",
 "contentType": "application/octet-stream",
 "timeCreated": "2018-04-05T19:21:44.033Z",
 "updated": "2018-04-05T19:21:44.033Z",
 "storageClass": "MULTI_REGIONAL",
 "timeStorageClassUpdated": "2018-04-05T19:21:44.033Z",
 "size": "59225004",
 "md5Hash": "BG1rEe0zy8oe6U/Udcg0NQ==",
 "mediaLink": "https://www.googleapis.com/download/storage/v1/b/tarapareddy/o/test2?generation=1522956104046841&alt=media",
 "crc32c": "w1oBgw==",
 "etag": "CPmx3Mzto9oCEAE="
}

but i want the object downloaded into my flow..their documentation shows to use CURL in the format below. how do i set the -o .?? i tried setting it in Attributes to send , but it didnt work.

curl -X GET \
    -H "Authorization: Bearer [OAUTH2_TOKEN]" \
    -o "[SAVE_TO_LOCATION]" \
    "https://www.googleapis.com/storage/v1/b/[BUCKET_NAME]/o/[OBJECT_NAME]?alt=media"

[#2]

I think you should be able to provide "Authorization" in Attributes to Send, and have an attribute "Authorization" with value "Bearer [OAUTH2_TOKEN]", with that value replaced with the token (you can use UpdateAttribute with Expression Language to set all that up). Then you should be able to use the URL that works with curl.

If that doesn't work, try an attribute with any name (say, "auth") with value "Authorization: Bearer [OAUTH2_TOKEN]" and include "auth" in the Attributes to Send property.

[#3]

Hi @mburgess 

I tried adding attribute name “auth” in Attributes to Send property and had this “auth” as a custom property in my invokeHTTP processor with value 'Authorization: Bearer ${access_token}' where the value of variable access_token gets populated from EvaluateJsonPath processor in front of my invokeHTTP, but somehow I am still getting unauthorized as the response.

Is there anything else that I’m missing for token to get passed on as a header in this call?

[#4]

@Matt Burgess,

Thanks for the reply , I am more concerned about the -o "[SAVE_TO_LOCATION]" , should I be sending it also in

Attributes to Send. can I send both (as comma separated or how).?? the CURL command shows options -H and -o ,can I add it to the attributes.?

Regards,

Sai

[#5]

You don't need to specify that parameter, in curl the -o lets you pick the destination of the output. In InvokeHttp the response will be in the content of the outgoing flow file. If you want to put that content into a file, you can use PutFile after InvokeHttp.

[#6]

@Matt Burgess ,

I am getting this error when I tried to send the Authorization from Attributes To Send..any idea why.??

InvokeHTTP[id=e6af6693-2428-3261-5418-a9cd3fc4c76b] Routing to Failure due to exception: java.lang.IllegalArgumentException: Unexpected char 0xa0 at 14 in header value: Authorization: Bearer ya29.GlyVBYcjf5qiEVyoPHJzDjf3ZLiXacV5Oaoaj817Tl-FzChWs9TSRc3vVNO8aMEj_HGHQMStSSJOwzQ3-WOygapsx8n4qmM-IVWFstZFtMa5Ld-47ZmKSClWEKFW5A: java.lang.IllegalArgumentException: Unexpected char 0xa0 at 14 in header value: Authorization: Bearer ya29.GlyVBYcjf5qiEVyoPHJzDjf3ZLiXacV5Oaoaj817Tl-FzChWs9TSRc3vVNO8aMEj_HGHQMStSSJOwzQ3-WOygapsx8n4qmM-IVWFstZFtMa5Ld-47ZmKSClWEKFW5A

[#7]

Hi @Matt Burgess ,

i replaced the space with %20 after Bearer , but now getting this response..

{ "error": { "errors": [ { "domain": "global", "reason": "authError", "message": "Invalid Credentials", "locationType": "header", "location": "Authorization" } ], "code": 401, "message": "Invalid Credentials" } }

where as if i try the url directly using , its working..any help.??

https://www.googleapis.com/storage/v1/b/tarapareddy/o/test2?access_token=${access_token}