Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Accessing parameters/properties with a Hive action

Highlighted

Accessing parameters/properties with a Hive action

Contributor

Folks, I tried to access properties from a job-xml file with Oozie in a Hive action.

(Because it does not work with the global settings, cf.: post)

So that is what I have tried so far:

1. Adding hive-site.xml and hive_param.xml via <job-xml> to the Hive action 2. Adding a property with <configuration> to the Hive action

In both cases I get the error that my variable cannot be resolved when I try to access it via <param>test=${test_var}</param>

As third option I added "test_var=testtest" to my job.properties. In this case I can access the variable within the Hive action.

What am I doing wrong in the first two cases?

Here is some code: Case 1

   ... 
    <action name="hive-insert">
          <hive xmlns="uri:oozie:hive-action:0.6">
	      <job-xml>hive-site.xml</job-xml>
	      <job-xml>hive_param.xml</job-xml>
              <script>${oozie_root}/test_insert.hql</script>
              <param>test=${test_var}</param>
           </hive>
          <ok to="end"/>
          <error to="kill"/>
      </action>
...

with hive_param.xml:

	      <configuration>
		   <property>
			<name>test_var</name>
			<value>testtest</value>
		   </property>
	      </configuration>

Case 2

... 
    <action name="hive-insert">
          <hive xmlns="uri:oozie:hive-action:0.6">
	      <configuration>
		   <property>
			<name>test_var</name>
			<value>testtest</value>
		   </property>
	      </configuration>
              <script>${oozie_root}/test_insert.hql</script>
              <param>test=${test_var}</param>
           </hive>
          <ok to="end"/>
          <error to="kill"/>
      </action> 
...		

I am really looking forward to your help since I have no clue what am I doing wrong here.

Thank you !

2 REPLIES 2

Re: Accessing parameters/properties with a Hive action

New Contributor

@Ken Jiiii You can just have one <job-xml> for one oozie action. That is why the case1 failed.

If you want to add properties or modify any property value just for your oozie hive action, you can follow the below steps

1.copy the hive-site.xml to tmp folders and add the properties or modify any property value to the hive-site.xml under tmp folders

2.Upload the modified hive-site.xml to workflow directory under HDFS

3.At your workflow, just add <job-xml>hive-site.xml</job-xml>.

By doing that, oozie hive action will pick the newly uploaded hive-site.xml which contains the property you want for the Oozie hive action.

If you want add properties or modify any property value for all your Oozie hive action, then you can just modify the hive-site.xml under /etc/oozie/conf/action-conf/hive on the Oozie host. Oozie will pick up hive-site.xml there by default if you do not have <job-xml> in your Oozie hive action.

Re: Accessing parameters/properties with a Hive action

Contributor

Hi @xin.wang,

Thanks for your reply. Okay let us consider this: My workflow.xml is located in ${oozie_root}/wf/ in HDFS and my hive-site.xml is copied from local /etc/hive/conf/ to ${oozie_root}/wf/hive/ in HDFS.

When I append

<property>
<name>test_var</name>
<value>testtest</value>
</property>

To my hive-site.xml in HDFS and add it via

<job-xml>${oozie_root}/hive/hive-site.xml</job-xml>

to my Hive action I can use the property like this:

<script>${oozie_root}/test_insert.hql</script>              

<param>test=${test_var}</param>

In my test_insert.hql script I can access the parameter with ${test}. Is that correct?

Shouldn't it be the same as in case 1 of my question? Since

"As of schema 0.3, multiple job-xml elements are allowed in order to specify multiple job.xml files." (Source)

Thanks for your help! Kind regards, Ken

------

By the way... That is what I tried and did not work:

<!-- workflow.xml -->

   <action name="hive-insert">
        <hive xmlns="uri:oozie:hive-action:0.6">
            <job-xml>${oozie_root}/wf/hive/hive-site.xml</job-xml>
            <script>${oozie_root}/wf/hive/test_insert.hql</script>        
            <param>test=${test_var}</param>
        </hive>
        <ok to="end"/>
        <error to="kill"/>
    </action>
<!-- hive/hive-site.xml -->
<configuration>
    <!-- ... content of original hive-site.xml -->
    <property>
        <name>test_var</name>
        <value>testtest</value>
    </property>
</configuration>


The Status of the Oozie Job is always failed with the error message:

variable [test_var] cannot be resolved with an EL function error code

What am I doing wrong here? :-(