- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Error while running hive script from oozie workflow
- Labels:
-
Apache Oozie
Created on 02-14-2014 02:59 PM - edited 09-16-2022 01:54 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I got the below error while running the oozie workflow....
ID : 0000004-140214173424343-oozie-oozi-W@hive-node
------------------------------------------------------------------------------------------------------------------------------------
Console URL : -
Error Code : JA020
Error Message : JA020: Could not load credentials of type [hcat] with name [hive_credentials]]; perhaps it was not defined in oozie-site.xml?
External ID : -
External Status : ERROR
Name : hive-node
Retries : 0
Tracker URI : -
Type : hive
Started : 2014-02-14 22:49 GMT
Status : ERROR
Ended : 2014-02-14 22:49 GMT
------------------------------------------------------------------------------------------------------------------------------------
Does anyone have an idea about th eissue ? am I missing something ?
Thanks
Created 02-18-2014 07:13 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey,
Couple of things:
1. This section is not necessary, it doesn't hurt anything, but you don't need it:
<configuration>
<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>
2. What version of CDH are you using?
3. Are you using CM?
It really looks like you still don't have the credentials configured in the oozie-site.xml. If you are using CM, that might be why, can you chec, the instructions below and configure the oozie-site.xml for credentials?
For the Oozie configuration:
1. If using CM, add an Oozie proxy to the core-site.xml for the Hive metastore server:
- Go to, "HDFS Service->Configuration->Service-Wide->Advanced-> Cluster-wide Configuration Safety Valve for core-site.xml"
2. Add:
<property> <name>hadoop.proxyuser.oozie.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.oozie.groups</name> <value>*</value> </property>
3. Restart the Hive metastore server.
4. Add the HCat credentials class to oozie-site.xml.
Edit the file and add:
<property> <name>oozie.credentials.credentialclasses</name> <value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> </property>
6. Add:
<property> <name>oozie.credentials.credentialclasses</name> <value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> </property>
7. Restart Oozie.
Created 02-14-2014 05:49 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you past your workflow.xml?
It sounds like you're trying to use the hcat/hive metastore crededentials stuff (i.e. a <credentials> section) without adding the Credential classes to oozie-site.xml:
i.e.
<property> <name>oozie.credentials.credentialclasses</name> <value> hcat=org.apache.oozie.action.hadoop.HCatCredentials, hbase=org.apache.oozie.action.hadoop.HbaseCredentials, hive2=org.apache.oozie.action.hadoop.Hive2Credentials </value> </property>
This page has more details:
http://archive.cloudera.com/cdh4/cdh/4/oozie/DG_UnifiedCredentialsModule.html
Created 02-18-2014 07:07 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
my workflow.xml
<workflow-app xmlns="uri:oozie:workflow:0.4" name="hive-wf">
<credentials>
<credential name='hive_credentials' type='hcat'>
<property>
<name>hcat.metastore.uri</name>
<value>thrift://hostname.com:9083</value>
</property>
<property>
<name>hcat.metastore.principal</name>
<value>hive/_HOST@PRINCIPAL.COM</value>
</property>
</credential>
</credentials>
<start to="hive-node"/>
<action name="hive-node" cred="hive_credentials">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>/user/someuser/hive-site.xml</job-xml>
<configuration>
<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>
<script>script.q</script>
</hive>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>
${wf:name()} Work Flow Process failed miserably.
Work Flow Id{wf:id()}
Work Flow Name{wf:name()}
Error Node{wf:lastErrorNode()}
Error Message{wf:errorMessage(wf:lastErrorNode())}
</message>
</kill>
<end name="end"/>
</workflow-app>
Created 02-18-2014 07:13 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey,
Couple of things:
1. This section is not necessary, it doesn't hurt anything, but you don't need it:
<configuration>
<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>
2. What version of CDH are you using?
3. Are you using CM?
It really looks like you still don't have the credentials configured in the oozie-site.xml. If you are using CM, that might be why, can you chec, the instructions below and configure the oozie-site.xml for credentials?
For the Oozie configuration:
1. If using CM, add an Oozie proxy to the core-site.xml for the Hive metastore server:
- Go to, "HDFS Service->Configuration->Service-Wide->Advanced-> Cluster-wide Configuration Safety Valve for core-site.xml"
2. Add:
<property> <name>hadoop.proxyuser.oozie.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.oozie.groups</name> <value>*</value> </property>
3. Restart the Hive metastore server.
4. Add the HCat credentials class to oozie-site.xml.
Edit the file and add:
<property> <name>oozie.credentials.credentialclasses</name> <value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> </property>
6. Add:
<property> <name>oozie.credentials.credentialclasses</name> <value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> </property>
7. Restart Oozie.
Created 02-18-2014 07:15 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes we are using CM.
CM version: Cloudera Enterprise 4.8.0
Created 02-18-2014 07:23 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks, can you follow the steps I provided for CM and see if that helps?
Created 02-18-2014 08:13 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created 02-18-2014 08:23 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
No problem! Glad it worked!
Created 06-03-2014 11:01 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, I am using CDH5.
I followed this to do the modification via CM Web console, but I cannot see the config file core-site.xml updated in the file system. So, where is the working config files for oozie ? hive ? and hdfs ? It seems that, the changes I done on CM Web console, will not be applied to the config files located at /CDH/etc/oozie/conf.dist or /CDH/etc/hive/conf.dist
Created 06-04-2014 07:05 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
AFter you apply the changes in CM you will need to re-deploy the client configs to get those changes reflected on the client machine...