question Re: How to dynamically set the value of a property in a processor when the flow is running? in Support Questions

How to dynamically set the value of a property in a processor when the flow is running?

sayak17 — Fri, 15 Jul 2022 05:32:33 GMT

I have a flow in which I make a call using InvokeHTTP. The http call returns a JSON response.

This JSON response looks like this:

{ "token_type": "Bearer", "expires_in": 3599, "ext_expires_in": 3599, "access_token": "eyJ0eXAiOiJKV1QiLCJub25jZSI6InAzVWN1NmRBcTNPRmVGbDM5cTVoZE9EaUpUSFRHT2hVVWdYTUNLMEsyVjQiLCJhbGciOiJSUzI1NiIsIng1dCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSIsImtpZCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSJ9.eyJhdWQiOiJodHRwczovL2dyYXBoLm1pY3Jvc29mdC5jb20iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9iZWJhN2VlNi05ZWRhLTQzZTEtOTY5OS0yMDIxYzVmOTI5NDEvIiwiaWF0IjoxNjU3NTUzNzI3LCJuYmYiOjE2NTc1NTM3MjcsImV4cCI6MTY1NzU1NzYyNywiYWlvIjoiRTJaZ1lMalI4cjV6ZWRSMHRxUkErWFdIV2QrZkJnQT0iLCJhcHBfZGlzcGxheW5hbWUiOiJhcHAtZm9yLW5pZmkiLCJhcHBpZCI6ImRlMjg0MjNmLTgxYmMtNDc5Ni1hYTU3LTIyYTM5NmJkNDliYSIsImFwcGlkYWNyIjoiMSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2JlYmE3ZWU2LTllZGEtNDNlMS05Njk5LTIwMjFjNWY5Mjk0MS8iLCJpZHR5cCI6ImFwcCIsIm9pZCI6IjAwN2EwNjkyLTg5ZjYtNDlmYi04ZDFhLTQwZGE3YjcwZjgwNSIsInJoIjoiMC5BVGdBNW42NnZ0cWU0VU9XbVNBaHhma3BRUU1BQUFBQUFBQUF3QUFBQUFBQUFBQTRBQUEuIiwicm9sZXMiOlsiU2l0ZXMuUmVhZC5BbGwiXSwic3ViIjoiMDA3YTA2OTItODlmNi00OWZiLThkMWEtNDBkYTdiNzBmODA1IiwidGVuYW50X3JlZ2lvbl9zY29wZSI6Ik5BIiwidGlkIjoiYmViYTdlZTYtOWVkYS00M2UxLTk2OTktMjAyMWM1ZjkyOTQxIiwidXRpIjoiZ3VTaG5FY0RYRWVrNGp2em1HNV9BQSIsInZlciI6IjEuMCIsIndpZHMiOlsiMDk5N2ExZDAtMGQxZC00YWNiLWI0MDgtZDVjYTczMTIxZTkwIl0sInhtc190Y2R0IjoxNTU2MjMyNzkzfQ.U1ryZqE389sKARkqQTRbFLAXAo0Ip1uaj0nEuqbKUxkswBVgMnrUbWZDJC8p6PvDSKDksMlvldXaiO6jQkrie7XYvcGGWCFSfJkH7g3VRHtD675O0NDN1yTem8mNqzKkCkDrlv5gTr7JSVFUKf155EQ-u1nCY-jwIps7MKAfpWqM393nC2uexC2tt0Fi0lVRE8NXfK_Tuk6_16a9mKATyyZE3JGBZHC1ZWQC47TcIWmmhtfMQAV3qguz-XXqeXxBtYT6LSQIJLQg5cdB1qxx4TMExmzE1U9gOScU3FPArDQQnfdzWMUH_JeYZQZG_P7JaFUWm6eSxES_YB1r9KHDMQ" }

I want to take the value of the access_token from this response, add a prefix "Bearer " to it, and plug it in to the value of a property(added by me) called "Authorization" in the next InvokeHttp processor.

How should I achieve this? As is evident from my requirement, the next Invokehttp processor is dependent on this value before it can make its own http call to a different endpoint.

Thanks

Re: How to dynamically set the value of a property in a processor when the flow is running?

linssab — Fri, 15 Jul 2022 08:00:35 GMT

Hey @sayak17.

You can do this in a few different ways.

In any case, the first thing to do is get the access-token value from the JSON response. You can achieve this with an ExecuteScript processor as suggested in this post: NiFi - convert everything in json to attributes, not one by one (i.e JsonToAttributes)

I have made the same script in Python if you want and it's available on github.

Now you have all the JSON values as attributes in your flowfile. If you want to get rid of attributes you won't need, you can either modify the script yourself or use an UpdateAttribuite processor.

To add the "Bearer" string to your access-token value, you can make use of expression language in the second InvokeHTTP processor. Click on the icon inside the processor configuration window to add the attribute "Authorization". In the value field just add the following:

${access-token:prepend("Bearer")}

This will take the value of the attribute access-token and prepend the string "Bearer" to it.

Re: How to dynamically set the value of a property in a processor when the flow is running?

sayak17 — Fri, 15 Jul 2022 09:11:25 GMT

Hello @linssab

Thankyou for that detailed response. I have one doubt remaining: how to use the updateattribute processor to get rid of attributes I don't need?

Re: How to dynamically set the value of a property in a processor when the flow is running?

linssab — Fri, 15 Jul 2022 09:37:19 GMT

You can use regular expressions to filter off the attributes you don't need.
Let's say that you don't want the attributes "expires_in" and "ext_expires_in" to go on with your flowfile. In the UpdateRecord processor you can use the following expression to remove them from the flowfile in the "Delete Attributes Expression" field:

^.*expires_in.*$

This will match any attribute that contains the string "expires_in" and remove it from the flowfile.

Regards,

Re: How to dynamically set the value of a property in a processor when the flow is running?

sayak17 — Mon, 18 Jul 2022 06:19:27 GMT

@linssab For some reason, my second invokehttp is not working. When I got hold of the error message, it says:

{"error":{"code":"InvalidAuthenticationToken","message":"CompactToken parsing failed with error code: 80049217","innerError":{"date":"2022-07-18T06:12:26","request-id":"dd8bb873-dfb8-4afe-81d8-7d19bdf61a1f","client-request-id":"dd8aa873-dfb8-4afe-81d8-7d89bdz81a5f"}}}

Is there any way, to see what request the invokehttp is exactly is making..let's says as curl format, or any format. Or atleast, is it possible to view what exactly it is sending in the headers?

That will help me to debug.

Re: How to dynamically set the value of a property in a processor when the flow is running?

sayak17 — Mon, 18 Jul 2022 06:42:53 GMT

Finally, it is working as intended. Both invokehttp processors are working.

This article worked for me: Chaining Multiple Http API's in Apache NiFi | Medium