Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 03-07-2017 08:33 AM
I get an error while running an Oozie workflow with Hive queries.
Here is the workflow
<workflow-app xmlns='uri:oozie:workflow:0.5' name='reporting_W_errorAuditHiveQueryExe'>
<start to="hive_report_fork"/>
<fork name="hive_report_fork">
<path start="hiveGSRLfile"/>
<path start="hiveNGSRLfile"/>
<path start="hiveNGsrlRAfile"/>
</fork>
<action name="hiveGSRLfile">
<hive xmlns='uri:oozie:hive-action:0.2'>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<script>${dir}/gsrlQery.hql</script>
<param>OutputDir=${jobOutput}</param>
</hive>
<ok to="joining"/>
<error to="joining"/>
</action>
<action name="hiveNGSRLfile">
<hive xmlns='uri:oozie:hive-action:0.2'>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<script>${dir}/nongsrlQuery.hql</script>
<param>OutputDir=${jobOutput}</param>
</hive>
<ok to="joining"/>
<error to="joining"/>
</action>
<action name="hiveNGsrlRAfile">
<hive xmlns='uri:oozie:hive-action:0.2'>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<script>${dir}/nongsrlRAQuery.hql</script>
<param>OutputDir=${jobOutput}</param>
</hive>
<ok to="joining"/>
<error to="joining"/>
</action>
<join name= "joining" to="Success"/>
<action name="Success">
<email xmlns="uri:oozie:email-action:0.1">
<to>${failureEmailToAddress}</to>
<subject>Success</subject>
<body>
The workflow ${wf:name()} with id ${wf:id()} failed
[${wf:errorMessage(wf:lastErrorNode())}].
</body>
</email>
<ok to="end" />
<error to="fail" />
</action>
<action name="failure">
<email xmlns="uri:oozie:email-action:0.1">
<to>${failureEmailToAddress}</to>
<subject>Failure</subject>
<body>
The workflow ${wf:name()} with id ${wf:id()} failed
[${wf:errorMessage(wf:lastErrorNode())}].
</body>
</email>
<ok to="end" />
<error to="fail" />
</action>
<kill name="fail">
<message>Workflow failed</message>
</kill>
<end name="end"/>
</workflow-app>
And here is the oozie properties file
oozie.wf.application.path=${deploymentPath}/workflows/errorAuditHiveQueryExe.xml
deploymentPath=/user/amin/deploy_178
jobTracker=localhost:8032
nameNode=hdfs://nameservice1
dir=${deploymentPath}/data-warehouse/temp
[email protected]
jobOutput=${dir}
oozie.use.system.libpath=trueHere is the error I get:
FAILED: SemanticException [Error 10072]: Database does not exist: testnamespace Intercepting System.exit(10072) Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [10072]
However the namespace exist and I can query the tables inside it.
What could be wrong here? Please help.
Created 03-07-2017 09:58 AM
After a tireless research on the internet I was able to crack the solution for the issue. I have added a configuration to use the metastore server for the Hive job and it worked. Here is what I did to the Hive action.
....
<hive xmlns='uri:oozie:hive-action:0.2'>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>hive.metastore.uris</name>
<value>thrift://10.155.1.63:9083</value>
</property>
</configuration>
<script>${dir}/gsrlQery.hql</script>
<param>OutputDir=${jobOutput}</param>
</hive>
.... Note: replace the hive metatore ip accordingly if you are trying to fix a similar problem. To get the metastore details check the hive-site.xml file located in /etc/hive/conf dir.
Credit: MapR
Created 03-07-2017 09:58 AM
After a tireless research on the internet I was able to crack the solution for the issue. I have added a configuration to use the metastore server for the Hive job and it worked. Here is what I did to the Hive action.
....
<hive xmlns='uri:oozie:hive-action:0.2'>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>hive.metastore.uris</name>
<value>thrift://10.155.1.63:9083</value>
</property>
</configuration>
<script>${dir}/gsrlQery.hql</script>
<param>OutputDir=${jobOutput}</param>
</hive>
.... Note: replace the hive metatore ip accordingly if you are trying to fix a similar problem. To get the metastore details check the hive-site.xml file located in /etc/hive/conf dir.
Credit: MapR