Get list of keys in NiFi DistributedMapCache

[#1]

Here is my use case -- I would like to use the DistributedMapCache to cache about 30k records that I read from an http service. Once per day, I'd like to update the cache, which includes removing items which are no longer returned by the http service. I had wanted to implement this by writing a custom processor to retrieve the list of keys from the DistributedMapCache, and remove any item which did not get returned by the http service. However, when I looked at the interface for DistributedMapCacheClient, the method to return the set of keys (keySet) throws the UnsupportedOperationException. Does anyone have any suggestions on how else I can implement this? I'm a newbie with NiFi -- any suggestions you have would be greatly appreciated! Thanks!

[#2 · Accepted solution]

The default implementation is to throw UnsupportedOperationException, but many/most/all of the subclasses override it. Check the reference impl, and since you are removing things, also check removeAndGet(), removeByPattern(), and removeByPatternAndGet() to see if they would help, they might save you the trouble of fetching all keys to remove some of them.

[#3]

Ah yes thank you! Indeed the reference impl does implement the keySet method. I found your dcache groovy script and attempted to modify it to call the keySet method to return the set of keys, but I don't know groovy and so far it's not working... any chance I can get you to add a "getkeys" option to your groovy script if it's not too difficult? Thank you so much for your help! 🙂

[#4]

dcachegroovy.txt Here you go! The command is called "keys", prints each key on its own line.

[#5]

You are awesome! Thank you!!!