Using the Hue examples in the Cloudera QuickStart5.2.0, but the configuration is not working.

[#1]

Ok, I am using the quick start VM version 5.2.0. THis iscludes Hue, Hive, Sqoop, and other tools. I have not changed any configuration files.

When I run Hive queries in the Hive editor, they run fine. Also when I execute this Hive SQL command:

• show tables;

It runs fine, the tables it shows are the same in the left hand column.

Then I create an Oozie workflow with a Hive script. The script contains only one command:

• show tables;

The workflow runs with no problem. The submitter is Cloudera.

Then I use Hue and downlaod all the samples for Oozie and the other parts of the framework. I submit the sample workflow that runs a Hive step. The workflow does not succeed. Here is the error:

     ava.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Here is more detail:

2014-11-20 15:06:06,926 WARN HiveActionExecutor:544 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[Hive] JOB[0000004-141120121239453-oozie-oozi-W] ACTION[0000004-141120121239453-oozie-oozi-W@Hive] Launcher exception: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

Here are the settings for the metastore. The file is in /etc/hive/config.dist/hive-stie.xml

<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://127.0.0.1/metastore?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>

I am trying to allow Oozie to aaccess the metastore. So I access mysql, but I do not know the password for the local instalation of MySQL. Here is what I have tried:

[cloudera@quickstart ~]$ mysql -u root -p
Enter password: 
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
[cloudera@quickstart ~]$ mysql -u root -p
Enter password: 
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
[cloudera@quickstart ~]$ mysql -u root
ERROR 1045 (28000): Access denied for user
 'root'@'localhost' (using password: NO)
 [cloudera@quickstart ~]$

Should I try to make the MySQL instance accessable to Oozie? What config file is Oozie using? I hear some people changing to a Derby database, but if Hue comes with the Quick Start, it should work with the Quick Start, right? Does the Horton Sandbox have the same problems?

[#2 · Accepted solution]

So I found the issue. The example has a Hive configuration hard-coded in it that uses Derby as the metastore. I'll get this fixed in the next release, but to work around it now you can overwrite it with the real Hive configuration as follows:

sudo cp /etc/hive/conf/hive-site.xml /usr/lib/hue/apps/oozie/examples/managed/hive/hive-config.xml
sudo cp /etc/hive/conf/hive-site.xml /usr/lib/hue/apps/oozie/examples/unmanaged/hive/hive-config.xml

# And if you already installed the example in Hue...

sudo -u hdfs hadoop fs -rm /user/hue/oozie/workspaces/managed/hive/hive-config.xml
sudo -u hdfs hadoop fs -copyFromLocal /etc/hive/conf/hive-site.xml /user/hue/oozie/workspaces/managed/hive/hive-config.xml

sudo -u hdfs hadoop fs -rm /user/hue/oozie/workspaces/unmanaged/hive/hive-config.xml
sudo -u hdfs hadoop fs -copyFromLocal /etc/hive/conf/hive-site.xml /user/hue/oozie/workspaces/unmanaged/hive/hive-config.xml

Thanks for the feedback!

[#3 · Accepted solution]

I didn't check any logs (other than the ones you already found). I view the configuration for that example in the Oozie app in Hue by going to edit it, and saw that it reference hive-config.xml (which caught my eye because as you noticed, the real copy is called hive-site.xml). So I searched for files with that name in the native file system and in Hadoop, and saw that those files were bundled in Hue and included a copy of the default Hive configuration that uses a Derby metastore:

find / -name hive-config.xml 2>/dev/null
sudo -u hdfs hadoop fs -ls -R | grep hive-config.xml

[#4]

Per the documentation, the password for all MySQL users including root, hive, hue and oozie is 'cloudera'. In addition to the snippet you posted from hive-site.xml, you should also see those credentials configured there:

  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>cloudera</value>
  </property>

Oozie's configuration is in /etc/oozie/conf, and it too appears to be correctly configure to talk to MySQL, but the log does indicate that it's trying to use Derby and failing. I'll need to look into why.

[#5 · Accepted solution]

So I found the issue. The example has a Hive configuration hard-coded in it that uses Derby as the metastore. I'll get this fixed in the next release, but to work around it now you can overwrite it with the real Hive configuration as follows:

sudo cp /etc/hive/conf/hive-site.xml /usr/lib/hue/apps/oozie/examples/managed/hive/hive-config.xml
sudo cp /etc/hive/conf/hive-site.xml /usr/lib/hue/apps/oozie/examples/unmanaged/hive/hive-config.xml

# And if you already installed the example in Hue...

sudo -u hdfs hadoop fs -rm /user/hue/oozie/workspaces/managed/hive/hive-config.xml
sudo -u hdfs hadoop fs -copyFromLocal /etc/hive/conf/hive-site.xml /user/hue/oozie/workspaces/managed/hive/hive-config.xml

sudo -u hdfs hadoop fs -rm /user/hue/oozie/workspaces/unmanaged/hive/hive-config.xml
sudo -u hdfs hadoop fs -copyFromLocal /etc/hive/conf/hive-site.xml /user/hue/oozie/workspaces/unmanaged/hive/hive-config.xml

Thanks for the feedback!

[#6]

Which log did you check and how did you check it?

[#7 · Accepted solution]

I didn't check any logs (other than the ones you already found). I view the configuration for that example in the Oozie app in Hue by going to edit it, and saw that it reference hive-config.xml (which caught my eye because as you noticed, the real copy is called hive-site.xml). So I searched for files with that name in the native file system and in Hadoop, and saw that those files were bundled in Hue and included a copy of the default Hive configuration that uses a Derby metastore:

find / -name hive-config.xml 2>/dev/null
sudo -u hdfs hadoop fs -ls -R | grep hive-config.xml

[#8 · Accepted solution]

One thing to mention.  If a person has already run a workflow, just running it again won't work, the old settings are still there.  More is needed.  

1. Use the Job Browser to find the id of the workflow, or the use the Oozie dashboard for workflows.

2. Select on the ID

3.  Select the Details tab, then click on the application path.

4. Now youare in the File Browser, select hive-config.xml and check the hive configurations in that file.