Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 12-06-2016 02:57 AM
I'm trying to use the NiFi GetHBase processor and so far all I've gotten is errors. I'm using the configuration settings out of /etc/hbase/conf/hive-site.xml which has the zookeeper quorum (master.fulldomain.name,edge.fulldomain.name, port number 2181, and zookeeper znode parent of /hbase-unsecure) all correctly specified. I noticed as I cut and pasted this that it's getting a connection refused. I'm on a 2 VM development system so made sure I could connect from the nifi VM to the master node which is running zookeeper and hbase / phoenix. I've created the table in HBase using Phoenix as JOBS.JOB_REQUEST. I'm referencing that table as JOBS:JOB_REQUEST in the GetHBase processor. Anyone have any thoughts as to what might be going on. It seems it would be a nifi server to HBase connection issue. Also, is there any way to either (a) increase the logging of a particular processor or (b) debug a processor that's baked into NiFi (not that I really want to, but I would really like to get this particular processor working).
Here's the log contents from /var/log/nifi/nifi-app.log related to this error:
2016-12-05 18:45:39,660 ERROR [StandardProcessScheduler Thread-3] org.apache.nifi.hbase.GetHBase GetHBase[id=bf278170-0158-1000-6fb5-ae093b1db722] GetHBase[id=bf278170-0158-1000-6fb5-ae093b1db722] failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.; processor will not be scheduled to run for 30 seconds: java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. 2016-12-05 18:45:39,662 ERROR [StandardProcessScheduler Thread-1] org.apache.nifi.engine.FlowEngine A flow controller task execution stopped abnormally java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_77] at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_77] at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100) ~[na:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150) [na:1.8.0_77] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77] Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.GeneratedMethodAccessor437.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1244) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1240) ~[na:na] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_77] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_77] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_77] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77] ... 2 common frames omitted Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_77] at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_77] at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_77] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648) ~[na:1.8.0_77] at java.nio.channels.SocketChannel.open(SocketChannel.java:189) ~[na:1.8.0_77] at org.apache.nifi.distributed.cache.client.StandardCommsSession.<init>(StandardCommsSession.java:49) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.createCommsSession(DistributedMapCacheClientService.java:234) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.leaseCommsSession(DistributedMapCacheClientService.java:249) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.withCommsSession(DistributedMapCacheClientService.java:303) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.get(DistributedMapCacheClientService.java:184) ~[na:na] at sun.reflect.GeneratedMethodAccessor438.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77] at org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:177) ~[na:na] at com.sun.proxy.$Proxy138.get(Unknown Source) ~[na:na] at org.apache.nifi.hbase.GetHBase.getState(GetHBase.java:483) ~[na:na] at org.apache.nifi.hbase.GetHBase.parseColumns(GetHBase.java:212) ~[na:na] ... 15 common frames omitted
Created 12-06-2016 04:19 PM
I finally tracked it down, the clue which I didn't notice until I pasted the error message above was that the error was getting a connection refused on the DistributedCacheClient. I set the properties in the GetHBase | DistributedCacheClient, but hadn't created the DistributedCacheServer in the global settings for NiFi. Once I did that, then I hit HBase errors (to make it more fun HBase happened to have been down while I was testing). But once I fixed that, everything started to work.
Created 12-06-2016 03:40 PM
Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_77] at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_77] at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_77] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648) ~[na:1.8.0_77] at java.nio.channels.SocketChannel.open(SocketChannel.java:189) ~[na:1.8.0_77] at org.apache.nifi.distributed.cache.client.StandardCommsSession.<init>(StandardCommsSession.java:49) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.createCommsSession(DistributedMapCacheClientService.java:234) ~[na:na] at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.leaseCommsSession(DistributedMapCacheClientService.java:249)
Actually, it looks like you have intra-NiFi problems. This is not trying to connect to HBase which you can verify from the stack trace.
Created 12-06-2016 04:19 PM
I finally tracked it down, the clue which I didn't notice until I pasted the error message above was that the error was getting a connection refused on the DistributedCacheClient. I set the properties in the GetHBase | DistributedCacheClient, but hadn't created the DistributedCacheServer in the global settings for NiFi. Once I did that, then I hit HBase errors (to make it more fun HBase happened to have been down while I was testing). But once I fixed that, everything started to work.