Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Error while running hive script from oozie workflow

avatar
Expert Contributor

I got the below error while running the oozie workflow....

ID : 0000004-140214173424343-oozie-oozi-W@hive-node
------------------------------------------------------------------------------------------------------------------------------------
Console URL       : -
Error Code        : JA020
Error Message     : JA020: Could not load credentials of type [hcat] with name [hive_credentials]]; perhaps it was not defined in oozie-site.xml?
External ID       : -
External Status   : ERROR
Name              : hive-node
Retries           : 0
Tracker URI       : -
Type              : hive
Started           : 2014-02-14 22:49 GMT
Status            : ERROR
Ended             : 2014-02-14 22:49 GMT
------------------------------------------------------------------------------------------------------------------------------------

Does anyone have an idea about th eissue ? am I missing something ?

Thanks

Em Jay
1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hey, 

 

Couple of things:

 

1.  This section is not necessary, it doesn't hurt anything, but you don't need it:

 

<configuration>

<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>

 

2.  What version of CDH are you using?

 

3.  Are you using CM?

 

It really looks like you still don't have the credentials configured in the oozie-site.xml.  If you are using CM, that might be why, can you chec, the instructions below and configure the oozie-site.xml for credentials?

 

 

 

 

For the Oozie configuration:


1.  If using CM, add an Oozie proxy to the core-site.xml for the Hive metastore server:

- Go to, "HDFS Service->Configuration->Service-Wide->Advanced-> Cluster-wide Configuration Safety Valve for core-site.xml"

2. Add: 

<property> 

<name>hadoop.proxyuser.oozie.hosts</name> 

<value>*</value> 

</property> 

<property> 

<name>hadoop.proxyuser.oozie.groups</name> 

<value>*</value> 

</property> 


3. Restart the Hive metastore server.


4. Add the HCat credentials class to oozie-site.xml.

Edit the file and add:

<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property> 

5. In addition, If using CM, go to "Oozie service->Configuration->Oozie Server(default)->Advanced-> Oozie Server Configuration Safety Valve for oozie-site.xml"

6. Add:
 
<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property> 

7.  Restart Oozie. 

 

View solution in original post

10 REPLIES 10

avatar
Rising Star

Can you past your workflow.xml?

 

It sounds like you're trying to use the hcat/hive metastore crededentials stuff (i.e. a <credentials> section) without adding the Credential classes to oozie-site.xml:
i.e.

   <property>
      <name>oozie.credentials.credentialclasses</name>
      <value>
         hcat=org.apache.oozie.action.hadoop.HCatCredentials,
         hbase=org.apache.oozie.action.hadoop.HbaseCredentials,
         hive2=org.apache.oozie.action.hadoop.Hive2Credentials
      </value>
   </property>

 

This page has more details:
http://archive.cloudera.com/cdh4/cdh/4/oozie/DG_UnifiedCredentialsModule.html

Software Engineer | Cloudera, Inc. | http://cloudera.com

avatar
Expert Contributor

my workflow.xml

 

 <workflow-app xmlns="uri:oozie:workflow:0.4" name="hive-wf">

<credentials>
<credential name='hive_credentials' type='hcat'>
<property>
<name>hcat.metastore.uri</name>
<value>thrift://hostname.com:9083</value>
</property>
<property>
<name>hcat.metastore.principal</name>
<value>hive/_HOST@PRINCIPAL.COM</value>
</property>
</credential>
</credentials>

<start to="hive-node"/>
<action name="hive-node" cred="hive_credentials">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>/user/someuser/hive-site.xml</job-xml>
<configuration>

<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>
<script>script.q</script>
</hive>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
  <message>
     ${wf:name()} Work Flow Process failed miserably.
        Work Flow Id{wf:id()}
           Work Flow Name{wf:name()}
              Error Node{wf:lastErrorNode()}
                 Error Message{wf:errorMessage(wf:lastErrorNode())}
                   </message>
                    </kill>
<end name="end"/>
</workflow-app>

Em Jay

avatar
Super Collaborator

Hey, 

 

Couple of things:

 

1.  This section is not necessary, it doesn't hurt anything, but you don't need it:

 

<configuration>

<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>

 

2.  What version of CDH are you using?

 

3.  Are you using CM?

 

It really looks like you still don't have the credentials configured in the oozie-site.xml.  If you are using CM, that might be why, can you chec, the instructions below and configure the oozie-site.xml for credentials?

 

 

 

 

For the Oozie configuration:


1.  If using CM, add an Oozie proxy to the core-site.xml for the Hive metastore server:

- Go to, "HDFS Service->Configuration->Service-Wide->Advanced-> Cluster-wide Configuration Safety Valve for core-site.xml"

2. Add: 

<property> 

<name>hadoop.proxyuser.oozie.hosts</name> 

<value>*</value> 

</property> 

<property> 

<name>hadoop.proxyuser.oozie.groups</name> 

<value>*</value> 

</property> 


3. Restart the Hive metastore server.


4. Add the HCat credentials class to oozie-site.xml.

Edit the file and add:

<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property> 

5. In addition, If using CM, go to "Oozie service->Configuration->Oozie Server(default)->Advanced-> Oozie Server Configuration Safety Valve for oozie-site.xml"

6. Add:
 
<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property> 

7.  Restart Oozie. 

 

avatar
Expert Contributor
CDH Version: cdh4.5.0

Yes we are using CM.

CM version: Cloudera Enterprise 4.8.0
Em Jay

avatar
Super Collaborator

Thanks, can you follow the steps I provided for CM and see if that helps?

avatar
Expert Contributor
awesome...the conf change resolved the issue. Thanks a lot !
Em Jay

avatar
Super Collaborator

No problem!  Glad it worked!

avatar
Explorer

Hi, I am using CDH5. 

 

I followed this to do the modification via CM Web console, but I cannot see the config file core-site.xml  updated in the file system. So, where is the working config files for oozie ?  hive ? and hdfs ?  It seems that, the changes I done on CM Web console, will not be applied to the config files located at /CDH/etc/oozie/conf.dist or /CDH/etc/hive/conf.dist 

avatar
Expert Contributor

AFter you apply the changes in CM you will need to re-deploy the client configs to get those changes reflected on the client machine...

Em Jay