Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Executing Pig action with hbase from oozie - Question on oozie-site.xml

Highlighted

Executing Pig action with hbase from oozie - Question on oozie-site.xml

Expert Contributor

I had an issue earlier with calling Pig action reading from Hbase table directly from oozie (in kerberized cluster).

(link to my previous question: https://community.hortonworks.com/questions/24181/executing-pig-from-oozie-to-read-hbase-table-in-ke...)

The issue was that the token was not getting passed properly. So I was suggested to use add hbase credentials in the oozie workflow and use it. Following are the credentials I have in my workflow.xml

       <credentials>
        <credential name="hcat_cred" type="hcat">
        <property>
                <name>hcat.metastore.uri</name>
                <value>${metastore}</value>
        </property>
        <property>
                <name>hcat.metastore.principal</name>
                <value>${metastore_principal}</value>
        </property>
        </credential>
        <credential name="hbase_cred" type="hbase">
        </credential>
        </credentials>

But the problem is my oozie-site.xml has only following defined. No Hbase credential class is present in my oozie-site.xml. I tried adding it in the properties under <configuration></configuration> in my workflow.xml but this did not work. I dont have access to modify oozie-site.xml.

I have project specific hive-site.xml. tez-site.xml added in workflow.xml in <job-xml> property. Is it possible to define project specific oozie-site.xml in <job-xml> tag and use in my workflow without modifying the common oozie-site.xml file?

And also I need to access both hive and Hbase in the same pig script. What is the syntax to pass both the credentials ?

<action name="test_pig" cred="hcat_cred" cred="hbase_cred" >
2 REPLIES 2

Re: Executing Pig action with hbase from oozie - Question on oozie-site.xml

Explorer
@R M

Trying adding below credentials classes in your workflow definition.

<credentials>

<credential type="hcat" name="my_auth">

<property>

<name>hcat.metastore.uri</name>

<value>${metaStore}</value>

</property>

<property>

<name>hcat.metastore.principal</name>

<value>${hiveMetaStore}</value>

</property>

</credential>

<credential type="hbase" name="hbaseauth"> </credential> </credentials>

In the action definition where you are having pig script add below property,

<action name="<<Pig action Name>>" cred="my_auth,hbaseauth">

Highlighted

Re: Executing Pig action with hbase from oozie - Question on oozie-site.xml

Expert Contributor

Thanks for your reply. I had the same property in my workflow.xml

<credential type="hbase" name="hbaseauth"> </credential>

But even then it did not work. The error I got when I added this in workflow was

ErrorType [ERROR], ErrorCode [JA020], Message [JA020: Could not load credentials of type [hbase] with name [hbase_creds]]; perhaps it was not defined in oozie-site.xml?]
org.apache.oozie.action.ActionExecutorException: JA020: Could not load credentials of type [hbase] with name [hbase_creds]]; perhaps it was not defined in oozie-site.xml?

My oozie-site.xml does not have hbase type defined in it. It has only hcat and hive2. I cant modify this oozie-site.xml since I dont have access, so is there any workaround without modifying the oozie-site.xml?

Will it work if I point to some local copy of oozie-site.xml and use it in job-xml property inside workflow? I use hive-site.xml and tez-site.xml like this in job-xml property, can oozie-site.xml be also used like this?

    <property>
      <name>oozie.credentials.credentialclasses</name>
      <value>hcat=org.apache.oozie.action.hadoop.HCatCredentials,hive2=org.apache.oozie.action.hadoop.Hive2Credentials</value>
    </property>	
Don't have an account?
Coming from Hortonworks? Activate your account here