Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

PutDistributedMapCache throws exception

PutDistributedMapCache throws exception

New Contributor

I created RedisConnectionPool and RedisDistributedMapCacheClientService controller service.

When I use PutDistributedMapCache processor, it throws below exception:

	2019-01-24 00:03:17,084 WARN [Timer-Driven Process Thread-6] o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistributedMapCache[id=7541bb66-0168-1000-b720-385f1ecc773c] due to uncaught Exception: java.lang.IllegalArgumentException: Option must not be null!
java.lang.IllegalArgumentException: Option must not be null!
at org.springframework.util.Assert.notNull(Assert.java:198)
at org.springframework.data.redis.connection.jedis.JedisStringCommands.set(JedisStringCommands.java:159)
at org.springframework.data.redis.connection.DefaultedRedisConnection.set(DefaultedRedisConnection.java:281)
at org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.lambda$put$3(RedisDistributedMapCacheClientService.java:191)
at org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:344)
at org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
at sun.reflect.GeneratedMethodAccessor823.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
at com.sun.proxy.$Proxy88.put(Unknown Source)
at org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDistributedMapCache.java:202)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

I used below property for PutDistributedMapCache processor:

Distributed Cache Service - RedisDistributedMapCacheClientService
Cache update strategy - Replace if present

Can anyone please help to resolve this issue ?

2 REPLIES 2

Re: PutDistributedMapCache throws exception

New Contributor

I was able to use the Redis Distributed Map with no problems on 1.7.1. After upgrading to 1.8.0 the exact same flow was riddled with errors. Clearly the Redis Nar bundle was not unit tested after sweeping changes. At any rate the below code fixes can be made to the 1.8.0 repo to fix the bug you are seeing. It will require a recompile and upload of the Nar of course.

# /nifi-nar-bundles/nifi-redis-bundle/nifi-redis-extensions/src/main/java/org/apache/nifi/redis/service/RedisDistributedMapCacheClientService.java
(Line 38)  +import org.springframework.data.redis.connection.RedisStringCommands;
(Line 192) -            redisConnection.set(kv.getKey(), kv.getValue(), Expiration.seconds(ttl), null);
(Line 192) +            redisConnection.set(kv.getKey(), kv.getValue(), Expiration.seconds(ttl), RedisStringCommands.SetOption.SET_IF_PRESENT);

# A second bug exists when trying to connect to Redis on a hostname other than 'localhost', which this fixes.
# nifi-redis-extensions/src/main/java/org/apache/nifi/redis/util/RedisUtils.java
(Line 274) +            connectionFactory.setHostName(jedisShardInfo.getHost());
Highlighted

Re: PutDistributedMapCache throws exception

This appears to have been addressed in https://issues.apache.org/jira/browse/NIFI-5795