Can Hive configuration be passed to Sqoop?

[#1]

Is there a way to pass Hive configuration to Sqoop that would be evaluated in the same fashion as '--hiveconf' when the actual Hive job is run to conduct the move task of data from landing to Hive table?

'-D' has no effect.

This is particularly needed when Hive on Tez is the default engine and Hive starts up a Tez container no matter what other configuration has been provided.

[#2]

@kkane

"This is particularly needed when Hive on Tez is the default engine and Hive starts up a Tez container no matter what other configuration has been provided"

Could you elaborate more ? Do you want Hive to not launch Tez container?

[#3]

Yes, I think that preventing the Tez container would fix the problems which are: Set the queue name and use delegation tokens for Sqoop when running in an Oozie shell.

The cluster default Hive execution engine is Tez.

[#4 · Accepted solution]

The script will be executed by calling the installed copy of hive on the machine where Sqoop is run. If you have multiple Hive installations, or hive is not in your$PATH, use the --hive-home option to identify the Hive installation directory. Sqoop will use $HIVE_HOME/bin/hive from here.

[#5]

this is probably not the answer you're looking for but my guess is it should, especially with Oozie, you need to pass hive-site.xml. You probably need to do something like this

<command>[SQOOP-COMMAND]</command>
            <arg>[SQOOP-ARGUMENT]</arg>
	<file>lib/hive-site.xml</file>

[#6]

This is the Oozie shell vice Sqoop action but I took that and attempted anyway not thinking that Oozie configuration would not have effect on Sqoop running in the shell that calls its own Hive setup but it did. So, I used the dist cache <files></files> like you did in the Sqoop action and it worked.

Thanks.

[#7]

great, I think it will be worth if you show screenshots in this thread for everyone else to use.