- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Oozie shell action - sqoop cant find hive-site.xml from shell script
- Labels:
-
Apache Oozie
Created on ‎04-02-2015 09:39 AM - edited ‎09-16-2022 02:25 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sqoop called a script invoked by a shell action keeps defaulting to derby metastore even though mysql is configured in hive-site-xml. I've tried a number of ways to specify the location of hive-site.xml but am missing the secret sauce. Hive-site.xml is in the workflow directory on hdfs co-located with the script.
Thanks
Created ‎04-06-2015 06:05 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1. If using CM, put a Hive gateway on every NodeManager/TaskTracker in the cluster, if not using CM, put the hive-site.xml in /etc/hive/conf on every NodeManager/TaskTracker.
2. Add "export HIVE_CONF_DIR=`pwd`" to the top of your shell script and then sqoop shoudl check the local directory for the hive-site.xml.
Created ‎04-06-2015 06:05 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1. If using CM, put a Hive gateway on every NodeManager/TaskTracker in the cluster, if not using CM, put the hive-site.xml in /etc/hive/conf on every NodeManager/TaskTracker.
2. Add "export HIVE_CONF_DIR=`pwd`" to the top of your shell script and then sqoop shoudl check the local directory for the hive-site.xml.
Created ‎04-06-2015 10:28 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I ended up going with option 2 which solved the immediate problem. Thanks.
