Reply
Contributor
Posts: 29
Registered: ‎08-01-2014

HBase credential in workflow with Kerberos

Hi,

 

I created a Oozie workflow based on a shell action where I call the Hbase shell to create a table

 

echo "create 'HBASE_TABLE', {NAME => 'ORIGINAL', COMPRESSION => 'SNAPPY' }, {NAME => 'ADDED', COMPRESSION => 'SNAPPY' }"  | hbase shell

 

 

When I schedule it on Oozie in a Kerberized Cluster, I have to set the credential for hbase and I have to specify the hbase-site.xml in the workflow. Below an example:

 

<workflow-app name="BDPH0_InitializeHbaseTable" xmlns="uri:oozie:workflow:0.4">
  <credentials>
    <credential name="hb" type="hbase">
    </credential>
  </credentials>
    <start to="CreateHBaseTable"/>
    <action name="CreateHBaseTable" cred="hb" >
        <shell xmlns="uri:oozie:shell-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <job-xml>/user/oozie/share/lib/configuration-file/hbase-site.xml</job-xml>
            <exec>sh</exec>
			<argument>CreateHBaseTable.sh</argument>
            <file>CreateHBaseTable.sh#CreateHBaseTable.sh</file>
           <file>/user/oozie/share/lib/configuration-file/hbase-site.xml</file>
        </shell>
        <ok to="end"/>
        <error to="kill"/>
    </action>
	<kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

 

When I launch this workflow, it fails with this error:

 

2014-09-23 16:25:56,895 ERROR org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server] USER[user] GROUP[-] TOKEN[-] APP[BDPH0_InitializeHbaseTable] JOB[0000049-140919133511133-oozie-oozi-W] ACTION[0000049-140919133511133-oozie-oozi-W@CreateHBaseTable] Error,
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
at org.apache.oozie.action.hadoop.HbaseCredentials.copyHbaseConfToJobConf(HbaseCredentials.java:60)
at org.apache.oozie.action.hadoop.HbaseCredentials.addtoJobConf(HbaseCredentials.java:49)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:990)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:850)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1071)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:283)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

 

 

Without the credential, the script doesn't create the table for authentication problem as it should be.

 

Which jar should I import ? where ? How ? 

 

Thanks

Alessio

Cloudera Employee
Posts: 35
Registered: ‎07-08-2013

Re: HBase credential in workflow with Kerberos

You need to copy or symlink the following jars

from /opt/cloudera/parcels/CDH/lib/hbase

to /opt/cloudera/parcels/CDH/lib/oozie/libserver

hbase-common.jar
hbase-client.jar
hbase-server.jar
hbase-protocol.jar
lib/netty-*Final.jar
lib/htrace-core.jar

 and restart Oozie.

 

If using packages, the above paths would be /usr/lib/hbase and /usr/lib/oozie/libserver instead.

 

We're planning on doing this for users out-of-the-box in a future CDH release.

Software Engineer | Cloudera, Inc. | http://cloudera.com
Announcements