question Re: Error while running hive script from oozie workflow in Support Questions

Error while running hive script from oozie workflow

Manikumar Juttukonda — Fri, 16 Sep 2022 08:54:04 GMT

I got the below error while running the oozie workflow....

ID : 0000004-140214173424343-oozie-oozi-W@hive-node
------------------------------------------------------------------------------------------------------------------------------------
Console URL       : -
Error Code        : JA020
Error Message     : JA020: Could not load credentials of type [hcat] with name [hive_credentials]]; perhaps it was not defined in oozie-site.xml?
External ID       : -
External Status   : ERROR
Name              : hive-node
Retries           : 0
Tracker URI       : -
Type              : hive
Started           : 2014-02-14 22:49 GMT
Status            : ERROR
Ended             : 2014-02-14 22:49 GMT
------------------------------------------------------------------------------------------------------------------------------------

Does anyone have an idea about th eissue ? am I missing something ?

Thanks

Re: Error while running hive script from oozie workflow

Robert K — Sat, 15 Feb 2014 01:49:58 GMT

Can you past your workflow.xml?

It sounds like you're trying to use the hcat/hive metastore crededentials stuff (i.e. a <credentials> section) without adding the Credential classes to oozie-site.xml:
i.e.

   <property>
      <name>oozie.credentials.credentialclasses</name>
      <value>
         hcat=org.apache.oozie.action.hadoop.HCatCredentials,
         hbase=org.apache.oozie.action.hadoop.HbaseCredentials,
         hive2=org.apache.oozie.action.hadoop.Hive2Credentials
      </value>
   </property>

This page has more details:
http://archive.cloudera.com/cdh4/cdh/4/oozie/DG_UnifiedCredentialsModule.html

Re: Error while running hive script from oozie workflow

Manikumar Juttukonda — Tue, 18 Feb 2014 15:07:26 GMT

my workflow.xml

<workflow-app xmlns="uri:oozie:workflow:0.4" name="hive-wf">

<credentials>
<credential name='hive_credentials' type='hcat'>
<property>
<name>hcat.metastore.uri</name>
<value>thrift://hostname.com:9083</value>
</property>
<property>
<name>hcat.metastore.principal</name>
<value>hive/_HOST@PRINCIPAL.COM</value>
</property>
</credential>
</credentials>

<start to="hive-node"/>
<action name="hive-node" cred="hive_credentials">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>/user/someuser/hive-site.xml</job-xml>
<configuration>

<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>
<script>script.q</script>
</hive>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>
     ${wf:name()} Work Flow Process failed miserably.
        Work Flow Id{wf:id()}
           Work Flow Name{wf:name()}
              Error Node{wf:lastErrorNode()}
                 Error Message{wf:errorMessage(wf:lastErrorNode())}
                   </message>
                    </kill>
<end name="end"/>
</workflow-app>

Re: Error while running hive script from oozie workflow

cconner — Tue, 18 Feb 2014 15:13:04 GMT

Hey,

Couple of things:

1. This section is not necessary, it doesn't hurt anything, but you don't need it:

<configuration>

<property>
<name>oozie.hive.defaults</name>
<value>/user/someuser/hive-default.xml</value>
</property>
</configuration>

2. What version of CDH are you using?

3. Are you using CM?

It really looks like you still don't have the credentials configured in the oozie-site.xml. If you are using CM, that might be why, can you chec, the instructions below and configure the oozie-site.xml for credentials?

For the Oozie configuration:

1. If using CM, add an Oozie proxy to the core-site.xml for the Hive metastore server:

- Go to, "HDFS Service->Configuration->Service-Wide->Advanced-> Cluster-wide Configuration Safety Valve for core-site.xml"

2. Add:

<property> 

<name>hadoop.proxyuser.oozie.hosts</name> 

<value>*</value> 

</property> 

<property> 

<name>hadoop.proxyuser.oozie.groups</name> 

<value>*</value> 

</property>

3. Restart the Hive metastore server.

4. Add the HCat credentials class to oozie-site.xml.

Edit the file and add:

<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property>

5. In addition, If using CM, go to "Oozie service->Configuration->Oozie Server(default)->Advanced-> Oozie Server Configuration Safety Valve for oozie-site.xml"

6. Add:

<property> 

<name>oozie.credentials.credentialclasses</name> 

<value>hcat=org.apache.oozie.action.hadoop.HCatCredentials</value> 

</property>

7. Restart Oozie.

Re: Error while running hive script from oozie workflow

Manikumar Juttukonda — Tue, 18 Feb 2014 15:15:31 GMT

CDH Version: cdh4.5.0

Yes we are using CM.

CM version: Cloudera Enterprise 4.8.0

Re: Error while running hive script from oozie workflow

cconner — Tue, 18 Feb 2014 15:23:18 GMT

Thanks, can you follow the steps I provided for CM and see if that helps?

Re: Error while running hive script from oozie workflow

Manikumar Juttukonda — Tue, 18 Feb 2014 16:13:44 GMT

awesome...the conf change resolved the issue. Thanks a lot !

Re: Error while running hive script from oozie workflow

cconner — Tue, 18 Feb 2014 16:23:52 GMT

No problem! Glad it worked!

Re: Error while running hive script from oozie workflow

PeterPan — Wed, 04 Jun 2014 06:01:46 GMT

Hi, I am using CDH5.

I followed this to do the modification via CM Web console, but I cannot see the config file core-site.xml updated in the file system. So, where is the working config files for oozie ? hive ? and hdfs ? It seems that, the changes I done on CM Web console, will not be applied to the config files located at /CDH/etc/oozie/conf.dist or /CDH/etc/hive/conf.dist

Re: Error while running hive script from oozie workflow

Manikumar Juttukonda — Wed, 04 Jun 2014 14:05:56 GMT

AFter you apply the changes in CM you will need to re-deploy the client configs to get those changes reflected on the client machine...

Re: Error while running hive script from oozie workflow

Senthi — Tue, 10 Jun 2014 16:22:52 GMT

Hey

In Step 4 are we editing the file /opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/etc/oozie/conf.dist/oozie-site.xml manually?

We are using CM.

I see changes in CM but not in the oozie-site.xml or on Oozie admin UI .

Getting error

[JA020: Could not load credentials of type [hbase] with name [hbase_credentials]]; perhaps it was not defined in oozie-site.xml?]

Thanks