Reply
Highlighted
New Contributor
Posts: 7
Registered: ‎08-21-2013

problem running hive query through webhcat

When I run a simple hive query through WebHcat, there is error threw,

 

Hive error occurred: {"error":"java.net.URISyntaxException: Illegal character in path at index 1: ${env.TEMPLETON_HOME}/share/webhcat/svr/lib/zookeeper-3.4.3.jar"}

 

This should be the same issue in here,

https://groups.google.com/a/cloudera.org/forum/#!msg/cdh-user/xPMdBZ3L2VI/DrE6rx2A0d0J

 

In bigtop I can manually modify hadoo-env.sh to export env.TEMPLETON_HOME to resolve this problem, but for CM 4.6 how can I set this variable? 

Cloudera Employee
Posts: 509
Registered: ‎07-30-2013

Re: problem running hive query through webhcat

Hi Bailly,

 

You should use HiveServer2 to run hive queries instead of WebHCat. HiveServer2 was expressly designed for this purpose.

 

In general, if you want to customize the enviornment, you should search for the relevant environment safety valve. Just search for "safety" in the Hive service in the search box on the left and you'll see several things come up. The one you want is "WebHCat Server Service Environment Safety Valve", where you can make your changes.

 

Thanks,

Darren

New Contributor
Posts: 7
Registered: ‎08-21-2013

Re: problem running hive query through webhcat

Thanks Darren!  If I want to use REST API to call Hive/Pig/MapReduce then I have to go with WebHcat, right? 

Cloudera Employee
Posts: 509
Registered: ‎07-30-2013

Re: problem running hive query through webhcat

Hi Bailly,

 

You should use Oozie for execution via REST. I'm not familiar with the details though. WebHCat is supposed to just be used for metadata queries (show tables, get table definition, create table, delete table, etc), but not hive/pig/mr execution.

 

If you don't require REST, then HS2 is a good choice for hive query execution. You can use the HS2 JDBC driver or the beeline client to talk to HS2.

 

Thanks,

Darren

Expert Contributor
Posts: 63
Registered: ‎08-06-2013

Re: problem running hive query through webhcat

for CM 4.6 how can I set this variable? 

 


1. Open the Cloudera Manager Admin Console, click a service and click Configuration.
2. Under Service-Wide, click Advanced.
3. Click the Value cell for the "Service Environment Safety Valve" and add TEMPLETON_HOME to the property. For example, enter the value TEMPLETON_HOME=/opt/templeton.
4. Click Save Changes.
5. In the Cloudera Manager Admin Console, for the service configured, click Actions and click Restart.

Cloudera Employee
Posts: 98
Registered: ‎07-31-2013

Re: problem running hive query through webhcat

I was working through this and I found that there are several steps necessary to get this working.  Once you have the safety valve that was mentioned in the previous reply set, then you still get some zookeeper errors because WebHcat defaults to zookeeper-3.4.3 and

CDH 4.3 uses zookeeper-3.4.5.  Then there were some other issues.  However, if you do the following, it should work:

 

Steps for WebHcat Server Safety Valve:

 

1. Open the Cloudera Manager Admin Console, click the Hive service and click Configuration.
2. Under WebHCat Server, click Advanced.
3. Click the Value cell for the "WebHCat Server Configuration Safety Valve for webhcat-site.xml", add the following:

 

  <property>

    <name>templeton.libjars</name>

    <value>${env.TEMPLETON_HOME}/share/webhcat/svr/lib/zookeeper-3.4.5-cdh4.3.0.jar</value>

    <description>Jars to add to the classpath.</description>

  </property>

  <property>

    <name>templeton.hive.path</name>

    <value>hive-0.10.0-cdh4.3.0.tar.gz/hive/bin/hive</value>

    <description>The path to the Hive executable.</description>

  </property>

 

4.  Make sure in the above that you enter the correct version for the "zookeeper-3.4.5-cdh4.3.0.jar" and "hive-0.10.0-cdh4.3.0.tar.gz" based on your CDH version.

5. Click Save Changes.

 

Steps for Service-Wide Environment Safety Valve:

 

1. Open the Cloudera Manager Admin Console, click the Hive service and click Configuration.
2. Under Service-Wide, click Advanced.
3. Click the Value cell for the "Service Environment Safety Valve" and add TEMPLETON_HOME to the property. For example, enter the value TEMPLETON_HOME=/opt/cloudera/parcels/CDH/lib/hcatalog (if using parcels, if using rpms, use /usr/lib/hcatalog).
4. Click Save Changes.
5. In the Cloudera Manager Admin Console, for the service configured, click Actions and click Restart.

  

 

HDFS Setup:

 

 

cd /opt/cloudera/parcels/CDH/lib (if using parcels, /usr/lib for RPMs)

tar -zcvf /tmp/hive-0.10.0-cdh4.3.0.tar.gz  (this file name must match the "hive-0.10.0-cdh4.3.0.tar.gz" above in the WebHCat Server Configuration Safety Valve)

 

sudo -u hdfs hadoop fs -mkdir /apps/templeton

sudo -u hdfs hadoop fs -put /tmp/hive-0.10.0-cdh4.3.0.tar.gz /apps/templeton

sudo -u hdfs hadoop fs -chown -R hive /apps/templeton

 

 

The above only handles hive.  If you need Pig or anything else, then you will have to do similar steps for each app.  Go here for an idea of what you need  "http://svn.apache.org/repos/asf/hive/hcatalog-historical/branches/branch-0.5/webhcat/svr/src/main/co...

 

You will probably need to setup:

 

templeton.pig.path

templeton.streaming.jar

templeton.hadoop

 

 

Also, I ran into issues with Kerberos that I'm looking into.  It gives an error about HTTP/_HOST@REALM not being able to impersonate USER@REALM.  I'll keep looking into this.

 

Hope this helps!

yue
New Contributor
Posts: 1
Registered: ‎07-21-2015

Re: problem running hive query through webhcat

Hi, cconner,

i also got the error like this 'HOST@REALM not being able to impersonate USER@REALM'.

Have you found the solution?