Member since
07-26-2017
1
Post
0
Kudos Received
0
Solutions
07-26-2017
10:13 AM
Hi According to this https://community.hortonworks.com/questions/103459/clarifications-on-state-management-within-nifi-pro.html and my research - I understand that DistributedMapCache is not actually distributed and it runs on individual nodes. If the node running the server fails then the data is gone. Also, it is a cache server so has an eviction strategy, though it gives the option of persistence directory but that does not solve anytime availability problem. When we want to store some temporary state then it may be good but for long term persistent state we should rather rely on Zookeeper for its distributed nature. Unfortunately, I could not find any processor for putting data in Zookeeper. Other option would be to use database or distributed storage like HDFS, S3 etc. Please correct me if I am wrong anywhere. PS: I have the same case where I want to get the data from an API and wants to store the time upto which I have already requested the data.
... View more