Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error while running hive script from oozie workflow

Solved Go to solution

Error while running hive script from oozie workflow

Expert Contributor

I got the below error while running the oozie workflow....

ID : 0000004-140214173424343-oozie-oozi-W@hive-node
------------------------------------------------------------------------------------------------------------------------------------
Console URL       : -
Error Code        : JA020
Error Message     : JA020: Could not load credentials of type [hcat] with name [hive_credentials]]; perhaps it was not defined in oozie-site.xml?
External ID       : -
External Status   : ERROR
Name              : hive-node
Retries           : 0
Tracker URI       : -
Type              : hive
Started           : 2014-02-14 22:49 GMT
Status            : ERROR
Ended             : 2014-02-14 22:49 GMT
------------------------------------------------------------------------------------------------------------------------------------

Does anyone have an idea about th eissue ? am I missing something ?

Thanks

Em Jay
1 ACCEPTED SOLUTION

Accepted Solutions

Re: Error while running hive script from oozie workflow

Expert Contributor

Hey, 

 

Couple of things:

 

1.  This section is not necessary, it doesn't hurt anything, but you don't need it:

 

<configuration>

<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>

 

2.  What version of CDH are you using?

 

3.  Are you using CM?

 

It really looks like you still don't have the credentials configured in the oozie-site.xml.  If you are using CM, that might be why, can you chec, the instructions below and configure the oozie-site.xml for credentials?

 

 

 

 

For the Oozie configuration:


1.  If using CM, add an Oozie proxy to the core-site.xml for the Hive metastore server:

- Go to, "HDFS Service->Configuration->Service-Wide->Advanced-> Cluster-wide Configuration Safety Valve for core-site.xml"

2. Add: 

<property> 

<name>hadoop.proxyuser.oozie.hosts</name> 

<value>*</value> 

</property> 

<property> 

<name>hadoop.proxyuser.oozie.groups</name> 

<value>*</value> 

</property> 


3. Restart the Hive metastore server.


4. Add the HCat credentials class to oozie-site.xml.

Edit the file and add:

<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property> 

5. In addition, If using CM, go to "Oozie service->Configuration->Oozie Server(default)->Advanced-> Oozie Server Configuration Safety Valve for oozie-site.xml"

6. Add:
 
<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property> 

7.  Restart Oozie. 

 

10 REPLIES 10

Re: Error while running hive script from oozie workflow

Contributor

Can you past your workflow.xml?

 

It sounds like you're trying to use the hcat/hive metastore crededentials stuff (i.e. a <credentials> section) without adding the Credential classes to oozie-site.xml:
i.e.

   <property>
      <name>oozie.credentials.credentialclasses</name>
      <value>
         hcat=org.apache.oozie.action.hadoop.HCatCredentials,
         hbase=org.apache.oozie.action.hadoop.HbaseCredentials,
         hive2=org.apache.oozie.action.hadoop.Hive2Credentials
      </value>
   </property>

 

This page has more details:
http://archive.cloudera.com/cdh4/cdh/4/oozie/DG_UnifiedCredentialsModule.html

Software Engineer | Cloudera, Inc. | http://cloudera.com

Re: Error while running hive script from oozie workflow

Expert Contributor

my workflow.xml

 

 <workflow-app xmlns="uri:oozie:workflow:0.4" name="hive-wf">

<credentials>
<credential name='hive_credentials' type='hcat'>
<property>
<name>hcat.metastore.uri</name>
<value>thrift://hostname.com:9083</value>
</property>
<property>
<name>hcat.metastore.principal</name>
<value>hive/_HOST@PRINCIPAL.COM</value>
</property>
</credential>
</credentials>

<start to="hive-node"/>
<action name="hive-node" cred="hive_credentials">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>/user/someuser/hive-site.xml</job-xml>
<configuration>

<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>
<script>script.q</script>
</hive>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
  <message>
     ${wf:name()} Work Flow Process failed miserably.
        Work Flow Id{wf:id()}
           Work Flow Name{wf:name()}
              Error Node{wf:lastErrorNode()}
                 Error Message{wf:errorMessage(wf:lastErrorNode())}
                   </message>
                    </kill>
<end name="end"/>
</workflow-app>

Em Jay

Re: Error while running hive script from oozie workflow

Expert Contributor

Hey, 

 

Couple of things:

 

1.  This section is not necessary, it doesn't hurt anything, but you don't need it:

 

<configuration>

<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>

 

2.  What version of CDH are you using?

 

3.  Are you using CM?

 

It really looks like you still don't have the credentials configured in the oozie-site.xml.  If you are using CM, that might be why, can you chec, the instructions below and configure the oozie-site.xml for credentials?

 

 

 

 

For the Oozie configuration:


1.  If using CM, add an Oozie proxy to the core-site.xml for the Hive metastore server:

- Go to, "HDFS Service->Configuration->Service-Wide->Advanced-> Cluster-wide Configuration Safety Valve for core-site.xml"

2. Add: 

<property> 

<name>hadoop.proxyuser.oozie.hosts</name> 

<value>*</value> 

</property> 

<property> 

<name>hadoop.proxyuser.oozie.groups</name> 

<value>*</value> 

</property> 


3. Restart the Hive metastore server.


4. Add the HCat credentials class to oozie-site.xml.

Edit the file and add:

<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property> 

5. In addition, If using CM, go to "Oozie service->Configuration->Oozie Server(default)->Advanced-> Oozie Server Configuration Safety Valve for oozie-site.xml"

6. Add:
 
<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property> 

7.  Restart Oozie. 

 

Re: Error while running hive script from oozie workflow

Expert Contributor
CDH Version: cdh4.5.0

Yes we are using CM.

CM version: Cloudera Enterprise 4.8.0
Em Jay

Re: Error while running hive script from oozie workflow

Expert Contributor

Thanks, can you follow the steps I provided for CM and see if that helps?

Re: Error while running hive script from oozie workflow

Expert Contributor
awesome...the conf change resolved the issue. Thanks a lot !
Em Jay

Re: Error while running hive script from oozie workflow

Expert Contributor

No problem!  Glad it worked!

Re: Error while running hive script from oozie workflow

New Contributor

Hi, I am using CDH5. 

 

I followed this to do the modification via CM Web console, but I cannot see the config file core-site.xml  updated in the file system. So, where is the working config files for oozie ?  hive ? and hdfs ?  It seems that, the changes I done on CM Web console, will not be applied to the config files located at /CDH/etc/oozie/conf.dist or /CDH/etc/hive/conf.dist 

Highlighted

Re: Error while running hive script from oozie workflow

Expert Contributor

AFter you apply the changes in CM you will need to re-deploy the client configs to get those changes reflected on the client machine...

Em Jay