Storing dynamic data in Apache NiFi

[#1]

Hi all,

I am a really beginer in Apache NiFi and I am trying to find the best way to do the following process.

I have an API that exposes an endpoint that provides me a token for next requests (available for few hours). I would like to store this token for general purpose, because I need other flows accessing to this token.

I would preffer not using Files or external Databases, so  I was wondering about using variables... but honestly... no idea if this is the best way.

So far, I know how to connect with the API, (InvokeHTTP), and getting the token (EvaluateJsonPath), but I don't know how to store and where (Variable?) and how to use it in another flow.

Current flow so far:

Store Token: GenerateFlowFile -> InvokeHTTP ->EvaluateJsonPath -> ??

New request using token: GenerateFlowFile -> InvokeHTTP (Here I need to access the token)

Can anyone help me?

Thank u so much!!

Javier

[#2]

Hello  @jdelarosa91,

First off welcome to Apache NIFI and its ecosystem.

You can accomplish this by using PutDistributedMapCache and FetchDistributedMapCache processors.

First, you must configure theDistributedMapCacheServer and DistributedMapCacheClientService controller services.

Then your dataflow should be like so:

Get token:

InvokeHttp > UpdateAttribute ( Assign an attribute for instance MyToken and give it value TOKEN ) > EvaluateJsonPath ( get the value you want ) > PutDistributeMapCache ( For cache entry identifier give it the attribute ${MyToken}  )

Now in the MapCache we have a key, value like so:

TOKEN / <the string you got from EvaluateJsonPath>

And when you need to grab the token:

GenerateFlowFile ( Configure an attribute MyToken give it value of TOKEN ) > FetchDistributeMapCache ( cache identifier ${MyToken} and "out identifier in attribute" give it a name to call your token with) > InvokeHttp ( Here when you need to use the token you can call it using nifi attribute ${the attribute with the token retrieved from you MapCache )

[#3]

Hi, can you clarify your answer a bit more. "PutDistributedMapCache" processor can store only Content of FlowFile, not key and an arbitrary attribute (from documentation: "Gets the content of a FlowFile and puts it to a distributed map cache"). 

So how is it possible to cache simple values like UUID in the flow without necessity of storing a “heavy” payload (content)?

Can you give an example of such configuration?

Can you give an example of such configuration?