Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HBase credential in workflow with Kerberos

Highlighted

HBase credential in workflow with Kerberos

Contributor

Hi,

 

I created a Oozie workflow based on a shell action where I call the Hbase shell to create a table

 

echo "create 'HBASE_TABLE', {NAME => 'ORIGINAL', COMPRESSION => 'SNAPPY' }, {NAME => 'ADDED', COMPRESSION => 'SNAPPY' }"  | hbase shell

 

 

When I schedule it on Oozie in a Kerberized Cluster, I have to set the credential for hbase and I have to specify the hbase-site.xml in the workflow. Below an example:

 

<workflow-app name="BDPH0_InitializeHbaseTable" xmlns="uri:oozie:workflow:0.4">
  <credentials>
    <credential name="hb" type="hbase">
    </credential>
  </credentials>
    <start to="CreateHBaseTable"/>
    <action name="CreateHBaseTable" cred="hb" >
        <shell xmlns="uri:oozie:shell-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <job-xml>/user/oozie/share/lib/configuration-file/hbase-site.xml</job-xml>
            <exec>sh</exec>
			<argument>CreateHBaseTable.sh</argument>
            <file>CreateHBaseTable.sh#CreateHBaseTable.sh</file>
           <file>/user/oozie/share/lib/configuration-file/hbase-site.xml</file>
        </shell>
        <ok to="end"/>
        <error to="kill"/>
    </action>
	<kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

 

When I launch this workflow, it fails with this error:

 

2014-09-23 16:25:56,895 ERROR org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server] USER[user] GROUP[-] TOKEN[-] APP[BDPH0_InitializeHbaseTable] JOB[0000049-140919133511133-oozie-oozi-W] ACTION[0000049-140919133511133-oozie-oozi-W@CreateHBaseTable] Error,
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
at org.apache.oozie.action.hadoop.HbaseCredentials.copyHbaseConfToJobConf(HbaseCredentials.java:60)
at org.apache.oozie.action.hadoop.HbaseCredentials.addtoJobConf(HbaseCredentials.java:49)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:990)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:850)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1071)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:283)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

 

 

Without the credential, the script doesn't create the table for authentication problem as it should be.

 

Which jar should I import ? where ? How ? 

 

Thanks

Alessio

1 REPLY 1

Re: HBase credential in workflow with Kerberos

Contributor

You need to copy or symlink the following jars

from /opt/cloudera/parcels/CDH/lib/hbase

to /opt/cloudera/parcels/CDH/lib/oozie/libserver

hbase-common.jar
hbase-client.jar
hbase-server.jar
hbase-protocol.jar
lib/netty-*Final.jar
lib/htrace-core.jar

 and restart Oozie.

 

If using packages, the above paths would be /usr/lib/hbase and /usr/lib/oozie/libserver instead.

 

We're planning on doing this for users out-of-the-box in a future CDH release.

Software Engineer | Cloudera, Inc. | http://cloudera.com
Don't have an account?
Coming from Hortonworks? Activate your account here