Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Get list of keys in NiFi DistributedMapCache

avatar
New Contributor
Here is my use case -- I would like to use the DistributedMapCache to cache about 30k records that I read from an http service. Once per day, I'd like to update the cache, which includes removing items which are no longer returned by the http service. I had wanted to implement this by writing a custom processor to retrieve the list of keys from the DistributedMapCache, and remove any item which did not get returned by the http service. However, when I looked at the interface for DistributedMapCacheClient, the method to return the set of keys (keySet) throws the UnsupportedOperationException. Does anyone have any suggestions on how else I can implement this? I'm a newbie with NiFi -- any suggestions you have would be greatly appreciated! Thanks!
1 ACCEPTED SOLUTION

avatar
Master Guru

The default implementation is to throw UnsupportedOperationException, but many/most/all of the subclasses override it. Check the reference impl, and since you are removing things, also check removeAndGet(), removeByPattern(), and removeByPatternAndGet() to see if they would help, they might save you the trouble of fetching all keys to remove some of them.

View solution in original post

7 REPLIES 7

avatar
Master Guru

The default implementation is to throw UnsupportedOperationException, but many/most/all of the subclasses override it. Check the reference impl, and since you are removing things, also check removeAndGet(), removeByPattern(), and removeByPatternAndGet() to see if they would help, they might save you the trouble of fetching all keys to remove some of them.

avatar
New Contributor

Ah yes thank you! Indeed the reference impl does implement the keySet method. I found your dcache groovy script and attempted to modify it to call the keySet method to return the set of keys, but I don't know groovy and so far it's not working... any chance I can get you to add a "getkeys" option to your groovy script if it's not too difficult? Thank you so much for your help! 🙂

avatar
Master Guru

dcachegroovy.txt Here you go! The command is called "keys", prints each key on its own line.

avatar
New Contributor

You are awesome! Thank you!!!

avatar
Contributor

Hi there, the script looks good and exactly what we are looking for. Just a quick question, when we try to implement it, we get the following error:

Dave0x1_0-1710327889113.png

We are using the invokescriptprocessor and configure like this

Dave0x1_1-1710327976758.png

What are we missing here 😉 

Thank you.

avatar
Community Manager

@Dave0x1 As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Contributor

Hi Diana, good idea. I'll create a new thread. Thank you.