Member since
03-31-2016
4
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
15017 | 04-08-2016 08:34 AM |
04-08-2016
08:34 AM
Thanks for the response , I have passed the keytb file in shell action section of workflow that solved the issue. Workflow.xml
<file>${keytabaccount}#${keytabaccount}</file> In script : kinit ${keytabaccount}@xxx.xxx.com-k -t ${keytabaccount}.keytab The above approach resolved the problem, Where i can access hive tables in shell script through hive2 beeline. Thanks
... View more
03-31-2016
04:07 AM
Hi,
Trying to call Hive commands inside shell scripts using oozie shell action, Shell script execute standalone but with oozie shell actionit fails to connect to hive metastore.
We are using CDH 5.4.8
Workflow.xml
<global>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapreduce.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
</global>
<credentials>
<credential name='hcat-creds' type='hcat'>
<property>
<name>hcat.metastore.uri</name>
<value>thrift://hostname.net:9083</value>
</property>
<property>
<name>hcat.metastore.principal</name>
<value>hive/_HOST@myhostname.com</value>
</property>
</credential>
</credentials>
<start to="file-compaction-processing" />
<action name="file-compaction-processing" cred='hcat-creds'>
<shell xmlns="uri:oozie:shell-action:0.3">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>${HiveOozieSiteXML}</job-xml>
<exec>${wfPath}/hive_compaction.sh</exec>
<env-var>HADOOP_USER_NAME=${wf:user()}</env-var>
<file>${wfPath}/hive_compaction.sh#${wfPath}/hive_compaction.sh</file>
<file>${HiveOozieSiteXML}#${HiveOozieSiteXML}</file>
<capture-output />
</shell>
<ok to="decision-file-compacted" />
<error to="fail" />
</action>
hive_compaction.sh
hive -e "show databases"
But oozie always fails with the below error when we use Hive cli ,
Caused by: javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=/var/lib/hive/metastore/metastore_db;create=true, username = MyUser. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: java.sql.SQLException: Failed to create database '/var/lib/hive/metastore/metastore_db'
When we use the beeline we are getting the kerberos auth problem, Can anyone please guide whether the approach of accessing hive through shell action works or not.
... View more
Labels: