Created on 06-12-2016 04:54 PM - edited 09-16-2022 03:24 AM
I did finished cloudera exercise 1 successfully on CDH 5 and now I am trying to replicate below same scenario in cloudera url on CDH 5:-
http://www.cloudera.com/developers/get-started-with-hadoop-tutorial/exercise-2.html
Exact command cloudera gave is ADD JAR {{lib_dir}}/hive/lib/hive-contrib.jar;
Problem is when i issue below command in HIVE query editor:-
ADD JAR /home/cloudera/lib/hive-contrib.jar;
Hive query editor says:-
Your query has the following error(s):
Error while processing statement: /home/cloudera/lib/hive-contrib.jar does not exist
There are 2 issues for which i need solution:-
1)I am unable to find hive/lib directory on CDH 5
2)How to import these jar on on Hive/lib directory
My main intention is to get output of below query after invalidate metadata command from Impala
select count(*),url from tokenized_access_logs where url like '%\/product\/%' group by url order by count(*) desc;
But though i see tokenized_access_logs created at left hand side of HUE but i see no results of above query so i doubt that reason is that add jar command was not executed successfully above hence i am not getting output,can someone suggest how do i successfully execute Add jar command
I have added the screenshot of my VM which shows tokenized_access_logs created but no data when i run
select count(*),url from tokenized_access_logs where url like '%\/product\/%' group by url order by count(*) desc;
Also added is the error when i ran jar add jar
Created 06-12-2016 10:06 PM
Created 06-12-2016 05:01 PM
to resolve this i copied hive-contrib.jar from google and copied using wget to /user/hive/lib/ then i ran query in Impala as
ADD JAR /user/hive/lib/hive-contrib.jar;
But below exception thrown
AnalysisException: Syntax error in line 1: ADD JAR /user/hive/lib/hive-contrib.jar ^ Encountered: ADD Expected: ALTER, COMPUTE, CREATE, DESCRIBE, DROP, EXPLAIN, GRANT, INSERT, INVALIDATE, LOAD, REFRESH, REVOKE, SELECT, SET, SHOW, TRUNCATE, USE, VALUES, WITH CAUSED BY: Exception: Syntax error
Created 06-12-2016 10:06 PM
Created 06-13-2016 07:14 AM
Created 06-13-2016 10:27 PM
Thanks Harsha , I mean it
Created 12-07-2016 09:53 AM
Created 03-26-2018 10:01 AM
I Found the jar file was in this directory:
[cloudera@quickstart lib]$ find ./ -name hive-contrib.jar
./hive/lib/hive-contrib.jar
./oozie/oozie-sharelib-mr1/lib/hive/hive-contrib.jar
./oozie/oozie-sharelib-yarn/lib/hive/hive-contrib.jar
[cloudera@quickstart lib]$ pwd
/usr/lib/hive/lib
[cloudera@quickstart lib]$ ls -ltr hive-contrib.jar
lrwxrwxrwx 1 root root 32 Oct 23 09:59 hive-contrib.jar -> hive-contrib-1.1.0-cdh5.13.0.jar
[cloudera@quickstart lib]$
then used the same
ADD JAR /usr/lib/hive/lib/hive-contrib.jar
and it worked.
Created on 09-22-2017 05:33 PM - edited 09-22-2017 05:49 PM
I have removed the ADD JAR command (not supported in Impala) for the Tutorial Exercise 2 and Impala at Hue.
It worked, but the data has not been parsed correctly so we should use Hive query editor.
Also there is note for HiveServer2 (https://www.cloudera.com/documentation/enterprise/5-6-x/topics/cm_mc_hive_udf.html)
We can find the library location in Linux:
[cloudera@quickstart hive]$ sudo su -
[root@quickstart ~]# find / -name hive-contrib.jar
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/filecache/1019/hive-contrib.jar
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/filecache/114/hive-contrib.jar
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/filecache/476/hive-contrib.jar
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/filecache/295/hive-contrib.jar
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/filecache/838/hive-contrib.jar
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/filecache/657/hive-contrib.jar
/usr/lib/hive/lib/hive-contrib.jar
/usr/lib/oozie/oozie-sharelib-mr1/lib/hive/hive-contrib.jar
/usr/lib/oozie/oozie-sharelib-yarn/lib/hive/hive-contrib.jar