Created on 05-05-2017 06:16 PM - edited 08-17-2019 07:08 PM
Hello,
While I was trying to setup the HBase Client Service (disabling and enabling a few times, to edit the properties), I now see the service is stuck in "Disabling" state; when I select the only option that is now available, "Remove", it would not let me delete it, it says "cannot be deleted because it is not disabled"; I tried creating a second Client Service, but that too ended up in the same situation.
What I noticed is, even when enabling it, it never went to the "Enabled" state, it was showing "Enabling" for state.
Our HDP environment is Kerberized, and so I did provide the necessary information in the properties for HBase Client service. Initially, I did not supply the ZooKepper info, because the documentation says it's required only if the Hadoop Configuration Files are not supplied. But when it was showing an error ("The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master."), I added the ZooKepper info in the properties.
As a side note, PutHDFS works perfectly fine with the same Hadoop Configuration Files, Principal, and Keytab;
Please suggest how to make this work, thanks.
Created 05-09-2017 09:20 PM
Causedby: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory
This class is a Phoenix class, when you install Phoenix it sets this class name somewhere in hbase-site.xml and then when NiFi creates the HBase client, the HBase client is reading hbase-site.xml and trying to instantiate this class which doesn't exist in the HBase client.
In Apache NiFi 1.1.x there is a property on the HBase client service for specifying the location of the phoenix-client.jar. Setting that should solve this.
Created 05-10-2017 01:42 PM
Correct, it wouldn't happen from regular restarts.
I don't know exactly, but its something like /usr/hdp/current/phoenix
Created 05-10-2017 03:19 PM
@Bryan Bende thank you.
Created 05-10-2017 07:21 PM
@Bryan Bende, placed the phoenix jar file in NiFi's work directory path and restarted NiFi instance; now seeing a different error in the log when I enable the HBase client service - "org.apache.nifi.StdErr java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z"
Failed to invoke @OnEnabled method due to java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z 2017-05-10 14:13:50,016 ERROR [NiFi logging handler] org.apache.nifi.StdErr [StandardProcessScheduler Thread-6] ERROR org.apache.nifi.controller.service.StandardControllerServiceNode - 2017-05-10 14:13:50,016 ERROR [NiFi logging handler] org.apache.nifi.StdErr java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z 2017-05-10 14:13:50,016 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.hadoop.security.UserGroupInformation.<init>(UserGroupInformation.java:623) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1200) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.hadoop.SecurityUtil.loginKerberos(SecurityUtil.java:52) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.hbase.HBase_1_1_2_ClientService.createConnection(HBase_1_1_2_ClientService.java:226) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.hbase.HBase_1_1_2_ClientService.onEnabled(HBase_1_1_2_ClientService.java:178) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.lang.reflect.Method.invoke(Method.java:498) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:348) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) 2017-05-10 14:13:50,019 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) 2017-05-10 14:13:50,020 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2017-05-10 14:13:50,020 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2017-05-10 14:13:50,020 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.lang.Thread.run(Thread.java:745) 2017-05-10 14:13:50,020 ERROR [NiFi logging handler] org.apache.nifi.StdErr [StandardProcessScheduler Thread-6] ERROR org.apache.nifi.controller.service.StandardControllerServiceNode - Failed to invoke @OnEnabled method of HBase_1_1_2_ClientService[id=102e119a-19a2-1409-671f-dddd93a063de] due to java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z