NiFi GetHBase connect refused

jeff_watson — Tue, 06 Dec 2016 10:57:01 GMT

I'm trying to use the NiFi GetHBase processor and so far all I've gotten is errors. I'm using the configuration settings out of /etc/hbase/conf/hive-site.xml which has the zookeeper quorum (master.fulldomain.name,edge.fulldomain.name, port number 2181, and zookeeper znode parent of /hbase-unsecure) all correctly specified. I noticed as I cut and pasted this that it's getting a connection refused. I'm on a 2 VM development system so made sure I could connect from the nifi VM to the master node which is running zookeeper and hbase / phoenix. I've created the table in HBase using Phoenix as JOBS.JOB_REQUEST. I'm referencing that table as JOBS:JOB_REQUEST in the GetHBase processor. Anyone have any thoughts as to what might be going on. It seems it would be a nifi server to HBase connection issue. Also, is there any way to either (a) increase the logging of a particular processor or (b) debug a processor that's baked into NiFi (not that I really want to, but I would really like to get this particular processor working).

Here's the log contents from /var/log/nifi/nifi-app.log related to this error:

2016-12-05 18:45:39,660 ERROR [StandardProcessScheduler Thread-3] org.apache.nifi.hbase.GetHBase GetHBase[id=bf278170-0158-1000-6fb5-ae093b1db722] GetHBase[id=bf278170-0158-1000-6fb5-ae093b1db722] failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.; processor will not be scheduled to run for 30 seconds: java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. 2016-12-05 18:45:39,662 ERROR [StandardProcessScheduler Thread-1] org.apache.nifi.engine.FlowEngine A flow controller task execution stopped abnormally java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_77] at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_77] at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100) ~[na:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150) [na:1.8.0_77] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77] Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.GeneratedMethodAccessor437.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1244) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1240) ~[na:na] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_77] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_77] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_77] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77] ... 2 common frames omitted Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_77] at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_77] at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_77] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648) ~[na:1.8.0_77] at java.nio.channels.SocketChannel.open(SocketChannel.java:189) ~[na:1.8.0_77] at org.apache.nifi.distributed.cache.client.StandardCommsSession.<init>(StandardCommsSession.java:49) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.createCommsSession(DistributedMapCacheClientService.java:234) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.leaseCommsSession(DistributedMapCacheClientService.java:249) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.withCommsSession(DistributedMapCacheClientService.java:303) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.get(DistributedMapCacheClientService.java:184) ~[na:na] at sun.reflect.GeneratedMethodAccessor438.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77] at org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:177) ~[na:na] at com.sun.proxy.$Proxy138.get(Unknown Source) ~[na:na] at org.apache.nifi.hbase.GetHBase.getState(GetHBase.java:483) ~[na:na] at org.apache.nifi.hbase.GetHBase.parseColumns(GetHBase.java:212) ~[na:na] ... 15 common frames omitted

Re: NiFi GetHBase connect refused

elserj — Tue, 06 Dec 2016 23:40:41 GMT

Caused by: java.net.ConnectException: Connection refused at 
sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_77] at 
sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_77] at 
sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_77] at 
sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648) 
~[na:1.8.0_77] at 
java.nio.channels.SocketChannel.open(SocketChannel.java:189) 
~[na:1.8.0_77] at 
org.apache.nifi.distributed.cache.client.StandardCommsSession.<init>(StandardCommsSession.java:49)
 ~[na:na] at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.createCommsSession(DistributedMapCacheClientService.java:234)
 ~[na:na] at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.leaseCommsSession(DistributedMapCacheClientService.java:249)

Actually, it looks like you have intra-NiFi problems. This is not trying to connect to HBase which you can verify from the stack trace.

Re: NiFi GetHBase connect refused

jeff_watson — Wed, 07 Dec 2016 00:19:58 GMT

I finally tracked it down, the clue which I didn't notice until I pasted the error message above was that the error was getting a connection refused on the DistributedCacheClient. I set the properties in the GetHBase | DistributedCacheClient, but hadn't created the DistributedCacheServer in the global settings for NiFi. Once I did that, then I hit HBase errors (to make it more fun HBase happened to have been down while I was testing). But once I fixed that, everything started to work.

question Re: NiFi GetHBase connect refused in Archives of Support Questions (Read Only)

NiFi GetHBase connect refused

Re: NiFi GetHBase connect refused

Re: NiFi GetHBase connect refused