question Re: NiFi application design : maintaining state in Support Questions

NiFi application design : maintaining state

Jarinek — Thu, 10 Feb 2022 13:26:39 GMT

Hello,

a question regarding application design in NiFi.

We have a REST resource which provides data changes since a unix timestamp, which is passed inside a json structure to be sent as http POST.
The major question is how and where to keep the state of the last unix timestamp and how this state is accessible to operation.

We should be able to reset the timestamp to any value to repeat calls. Also we should be able set increment, which es added to the timestmap in the next call.

Similary, we have a related issue ina use case, which access a database. We generate a query in GenerateTableFetch and set the property "Maximum-value Columns".
Here the state is maintened by the processor, but it pretty much unclear how to set the state to specific stating value. To clear the state is obviously not appropriate.

Best regards
Jaro

Re: NiFi application design : maintaining state

steven-matison — Thu, 10 Feb 2022 14:04:04 GMT

@Jarinek you should check out Distributed Map Cache features of NiFi. Here is a great post to get started:

https://community.cloudera.com/t5/Community-Articles/Working-with-a-NiFi-DistributedMapCache/ta-p/248370

Re: NiFi application design : maintaining state

DianaTorres — Tue, 22 Feb 2022 19:23:51 GMT

@Jarinek Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. If you are still experiencing the issue, can you provide the information @steven-matison has requested?