Working with a NiFi DistributedMapCache

[#1]

In NiFi/HDF, it is possible to create a kind of lookup table of key/value pairs using a DistributedMapCacheServer. The DistributedMapCacheServer is used by various processors such as GetHBase and DetectDuplicate. It can also be leveraged by users in their flows, with the PutDistributedMapCache and FetchDistributedMapCache processors, by specifying a corresponding DistributedMapCacheClientService.

Sometimes, however, it might be the case that the user would like to interact with the DistributedMapCacheServer programmatically (and external to NiFi), say for removing specific entries, inserting/populating entries, etc. To that end I have written a Groovy script (dcachegroovy.txt, rename to dcache.groovy) to allow manipulation of the entries in a DistributedMapCacheServer from the command-line, e.g.

The usage is as follows:

Usage: groovy dcache.groovy <hostname> <port> <command> <args> 

Where <command> is one of the following: 

get: Retrieves the values for the keys (provided as arguments) 
remove: Removes the keys (specified as arguments) 
put: Sets the given keys to the given values, specified as arguments in the form: key1 value1 key2 value2 ... keyN valueN

So to insert entries "a = Hello" and "b = World" (assuming a local DistributedMapCacheServer at the default port), you can enter:

groovy dcache.groovy localhost 4557 put a Hello b World

Which outputs the following:

Set a = Hello
Set b = World

Then to retrieve the values:

groovy dcache.groovy localhost 4557 get a b

Which gives:

a = Hello
b = World

To remove an entry:

groovy dcache.groovy localhost 4557 remove b

Giving:

Removed b

Trying the get again for both values (where b no longer exists):

groovy dcache.groovy localhost 4557 get a b

Gives:

a = Hello
b =

This script can be used to pre-populate, clear, or inspect a DistributedMapCacheServer. I'd be interested to hear if you try it, whether you find it useful or not, and of course all suggestions for improvements are welcome. Cheers!

[#2]

Is it possible to clear the cache completely by not providing any keys?
Or how does the script behave in that case?

[#3]

Very interesting article, well done !

Have you done something similar with jython?

I'm currently trying to use an execute script, with jython, to put key/values onto the distributed map cache but can't find a way to do it...

BR,

Paulo Rente