Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Passing Global Properties to Hive Action Parameter

Solved Go to solution
Highlighted

Passing Global Properties to Hive Action Parameter

Contributor

Hey guys,

I would like to pass properties defined in the <global> section by <job-xml> or <configuration> to a Hive Action parameter. But I cannot manage to do so. Example config:

<workflow>
<global>
<job-xml>hive_params.xml</job-xml>
</global>
...
<hive 0.6>
<script>h_script.hql</script>
<param>test=${test_from_hive_params_xml}</param>
</hive>

How can I achive this properly without changing my hive-site.xml? Or to pass a second xml to Hive but I do not want to change my hive-site.xml.

This is needed since I do not want to type all parameters by hand and pass it via the global section.

Looking forward to your help!

Thanks and regards

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Passing Global Properties to Hive Action Parameter

@Ken Jiiii

You could create another file and call it from h_script.hql, or you just add lines to h_script.hql like the following:

This set statements will override the global settings for that specific job session.

Example of line:

set mapred.reduce.tasks=32;

Almost anything that is Hive global environments XML is game for override at session level.

Look at this post: https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer....

This shows a lot of these set statements which you can include in your set_h_script.hql invoked from h_script.hql or in the block added the beginning of your h_script.hql. Personally, I prefer a separate file invoked from my job script.

1 REPLY 1

Re: Passing Global Properties to Hive Action Parameter

@Ken Jiiii

You could create another file and call it from h_script.hql, or you just add lines to h_script.hql like the following:

This set statements will override the global settings for that specific job session.

Example of line:

set mapred.reduce.tasks=32;

Almost anything that is Hive global environments XML is game for override at session level.

Look at this post: https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer....

This shows a lot of these set statements which you can include in your set_h_script.hql invoked from h_script.hql or in the block added the beginning of your h_script.hql. Personally, I prefer a separate file invoked from my job script.