Community Articles

Find and share helpful community-sourced technical articles.
Labels (1)
avatar
Master Guru

In NiFi/HDF, it is possible to create a kind of lookup table of key/value pairs using a DistributedMapCacheServer. The DistributedMapCacheServer is used by various processors such as GetHBase and DetectDuplicate. It can also be leveraged by users in their flows, with the PutDistributedMapCache and FetchDistributedMapCache processors, by specifying a corresponding DistributedMapCacheClientService.

Sometimes, however, it might be the case that the user would like to interact with the DistributedMapCacheServer programmatically (and external to NiFi), say for removing specific entries, inserting/populating entries, etc. To that end I have written a Groovy script (dcachegroovy.txt, rename to dcache.groovy) to allow manipulation of the entries in a DistributedMapCacheServer from the command-line, e.g.

The usage is as follows:

Usage: groovy dcache.groovy <hostname> <port> <command> <args> 

Where <command> is one of the following: 

get: Retrieves the values for the keys (provided as arguments) 
remove: Removes the keys (specified as arguments) 
put: Sets the given keys to the given values, specified as arguments in the form: key1 value1 key2 value2 ... keyN valueN

So to insert entries "a = Hello" and "b = World" (assuming a local DistributedMapCacheServer at the default port), you can enter:

groovy dcache.groovy localhost 4557 put a Hello b World

Which outputs the following:

Set a = Hello
Set b = World

Then to retrieve the values:

groovy dcache.groovy localhost 4557 get a b

Which gives:

a = Hello
b = World

To remove an entry:

groovy dcache.groovy localhost 4557 remove b

Giving:

Removed b

Trying the get again for both values (where b no longer exists):

groovy dcache.groovy localhost 4557 get a b

Gives:

a = Hello
b =

This script can be used to pre-populate, clear, or inspect a DistributedMapCacheServer. I'd be interested to hear if you try it, whether you find it useful or not, and of course all suggestions for improvements are welcome. Cheers!

16,372 Views
Comments
avatar
New Contributor

Is it possible to clear the cache completely by not providing any keys?
Or how does the script behave in that case?

avatar
New Contributor

Very interesting article, well done !


Have you done something similar with jython?


I'm currently trying to use an execute script, with jython, to put key/values onto the distributed map cache but can't find a way to do it...


BR,

Paulo Rente