Skip to main content
mburgess
Gold ⭐️⭐️⭐️
Gold ⭐️⭐️⭐️
December 16, 2016

Working with a NiFi DistributedMapCache

  • October 5, 2026 at 9:00 AM
  • 2 replies
  • 19113 views
  • Translate All

In NiFi/HDF, it is possible to create a kind of lookup table of key/value pairs using a DistributedMapCacheServer. The DistributedMapCacheServer is used by various processors such as GetHBase and DetectDuplicate. It can also be leveraged by users in their flows, with the PutDistributedMapCache and FetchDistributedMapCache processors, by specifying a corresponding DistributedMapCacheClientService.

Sometimes, however, it might be the case that the user would like to interact with the DistributedMapCacheServer programmatically (and external to NiFi), say for removing specific entries, inserting/populating entries, etc. To that end I have written a Groovy script (dcachegroovy.txt, rename to dcache.groovy) to allow manipulation of the entries in a DistributedMapCacheServer from the command-line, e.g.

The usage is as follows:

Usage: groovy dcache.groovy <hostname> <port> <command> <args>

Where <command> is one of the following:

get: Retrieves the values for the keys (provided as arguments)
remove: Removes the keys (specified as arguments)
put: Sets the given keys to the given values, specified as arguments in the form: key1 value1 key2 value2 ... keyN valueN

So to insert entries "a = Hello" and "b = World" (assuming a local DistributedMapCacheServer at the default port), you can enter:

groovy dcache.groovy localhost 4557 put a Hello b World

Which outputs the following:

Set a = Hello
Set b = World

Then to retrieve the values:

groovy dcache.groovy localhost 4557 get a b

Which gives:

a = Hello
b = World

To remove an entry:

groovy dcache.groovy localhost 4557 remove b

Giving:

Removed b

Trying the get again for both values (where b no longer exists):

groovy dcache.groovy localhost 4557 get a b

Gives:

a = Hello
b =

This script can be used to pre-populate, clear, or inspect a DistributedMapCacheServer. I'd be interested to hear if you try it, whether you find it useful or not, and of course all suggestions for improvements are welcome. Cheers!

2 replies

w_ederer
Silver ⭐️
April 8, 2019

Is it possible to clear the cache completely by not providing any keys?
Or how does the script behave in that case?

Translate
paulo_rente_92
Silver ⭐️
June 7, 2019

Very interesting article, well done !


Have you done something similar with jython?


I'm currently trying to use an execute script, with jython, to put key/values onto the distributed map cache but can't find a way to do it...


BR,

Paulo Rente

Translate