Support Questions

Find answers, ask questions, and share your expertise

Hive beeline throws ClassNotFoundException after installing atlas

avatar
Rising Star

Hello Team,

We have a HDP 2.6 cluster with HIVE in HA using zookeeper discovery enabled. Before installing atlas on cluster All comands related to hive through beeline was working.

Post installing atlas through ambari when i run simple show tables command it throws below error.

[ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; Error: Error while processing statement: FAILED: Hive Internal Error: java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook) (state=08S01,code=12)

Kindly help me to fix the issue.

- Vijay Mishra

11 REPLIES 11

avatar
Super Guru

@Vijay Mishra,

Did you enable Atlas Hook under hive. If not, then please enable Hive hook and retry.

Go to Ambari -> Hive -> Configs -> Advanced -> Advanced hive-env

Please refer screenshot

56588-screen-shot-2018-02-05-at-34517-pm.png

Thanks,

Aditya

avatar
Rising Star

atlas-hook-hive.png

@ Aditya

Its enabled, Attached screenshot for the same.

- Vijay Mishra

avatar
Super Guru

@Vijay Mishra,

Can you please run the command and attach the output here

[root@sss ~]# ll /usr/bin/beeline
lrwxrwxrwx. 1 root root 40 Dec  6 12:29 /usr/bin/beeline -> /usr/hdp/current/hive-client/bin/beeline

[root@sss ~]# cat /usr/hdp/current/hive-client/bin/beeline (cat the content of your symlink)

Thanks,

Aditya

avatar
Rising Star

@Aditya,

[root@vijayhdf-2 hive]# ll /usr/bin/beeline lrwxrwxrwx 1 root root 40 Jan 30 08:47 /usr/bin/beeline -> /usr/hdp/current/hive-client/bin/beeline [root@vijayhdf-2 hive]# cat /usr/hdp/current/hive-client/bin/beeline #!/bin/bash if [ -d "/usr/hdp/2.6.3.0-235/atlas/hook/hive" ]; then if [ -z "${HADOOP_CLASSPATH}" ]; then export HADOOP_CLASSPATH=/usr/hdp/2.6.3.0-235/atlas/hook/hive/* else export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:/usr/hdp/2.6.3.0-235/atlas/hook/hive/* fi fi BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default} [ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hbase ] && . ${BIGTOP_DEFAULTS_DIR}/hbase export HIVE_HOME=${HIVE_HOME:-/usr/hdp/2.6.3.0-235/hive} export HADOOP_HOME=${HADOOP_HOME:-/usr/hdp/2.6.3.0-235/hadoop} export ATLAS_HOME=${ATLAS_HOME:-/usr/hdp/2.6.3.0-235/atlas} HCATALOG_JAR_PATH=/usr/hdp/2.6.3.0-235/hive-hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1000.2.6.3.0-235.jar:/usr/hdp/2.6.3.0-235/hive-hcatalog/share/hcatalog/hive-hcatalog-server-extensions-1.2.1000.2.6.3.0-235.jar:/usr/hdp/2.6.3.0-235/hive-hcatalog/share/webhcat/java-client/hive-webhcat-java-client-1.2.1000.2.6.3.0-235.jar if [ -z "${HADOOP_CLASSPATH}" ]; then export HADOOP_CLASSPATH=${HCATALOG_JAR_PATH} else export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:${HCATALOG_JAR_PATH} fi exec "${HIVE_HOME}/bin/beeline.distro" "$@" [root@vijayhdf-2 hive]#

avatar
Rising Star

@Aditya,

For verify purpose i just disable the atlas hook from hive and ran the beeline and its working.

[ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; +-----------+--+ | tab_name | +-----------+--+ +-----------+--+ No rows selected (2.28 seconds) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi>

Post which i reenable atlas hook into hive and ran the beeline again and got the same error.

[ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; Error: Error while processing statement: FAILED: Hive Internal Error: java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook) (state=08S01,code=12) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi>

avatar
Super Guru

@Vijay Mishra,

Do you have atlas client installed on the node from where you are running beeline. If not, can you please run the command and try again.

curl -k -u {ambari-username}:{ambari-password} -H "X-Requested-By:ambari" -i -X PUT -d '{"HostRoles": {"state": "INSTALLED"}}' http://{ambari-host}:{ambari-port}/api/v1/clusters/{clustername}/hosts/{hostname}/host_components/AT...

Replace ambari-username, ambari-password, ambari-host, ambari-port, clustername with their respective values in the url.

Replace {hostname} with the hostname where you are running beeline client.

Thanks,

Aditya

avatar
Rising Star

@Aditya,

Atlas client installed on all 4 nodes of cluster. Still i ran the command provided by you. Below snap for the same.

[root@vijayhdf-1 hive]# curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"HostRoles": {"state": "INSTALLED"}}' http://vijayhdf-1.novalocal:8080/api/v1/clusters/hdpdev/hosts/vijayhdf-1.novalocal/host_components/A... HTTP/1.1 202 Accepted X-Frame-Options: DENY X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff Cache-Control: no-store Pragma: no-cache Set-Cookie: AMBARISESSIONID=n8czyjmtzvhzevwgenc583h5;Path=/;HttpOnly Expires: Thu, 01 Jan 1970 00:00:00 GMT User: admin Content-Type: text/plain Vary: Accept-Encoding, User-Agent Content-Length: 149 { "href" : "http://vijayhdf-1.novalocal:8080/api/v1/clusters/hdpdev/requests/122", "Requests" : { "id" : 122, "status" : "Accepted" } }[root@vijayhdf-1 hive]#

I restarted Hive and atlas and ran the beeline but it throws error again.

[ajay@vijayhdf-1 ~]$ hostname -f vijayhdf-1.novalocal [ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; Error: Error while processing statement: FAILED: Hive Internal Error: java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook) (state=08S01,code=12) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi>

- Vijay Mishra

avatar
Rising Star

@Aditya

I explored few more thing.

Updated below property as mentioned below and restarted hive and atlas.

From:

hive.exec.post.hooks-org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook

To:

hive.exec.post.hooks-org.apache.hadoop.hive.ql.hooks.ATSHook

All commands in beeline started getting executed but in Atlas UI i am unable to see hive table.

[ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; +-----------+--+ | tab_name | +-----------+--+ | emp | +-----------+--+ 1 row selected (2.232 seconds) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> select * from emp; +----------+-----------+-------------+------------------+--+ | emp.eid | emp.name | emp.salary | emp.destination | +----------+-----------+-------------+------------------+--+ +----------+-----------+-------------+------------------+--+ No rows selected (1.519 seconds)

0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> LOAD DATA INPATH 'hdfs://hdpdev/user/ajay/sample.txt' INTO TABLE emp;

0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> select * from emp; +----------+--------------+-------------+------------------+--+ | emp.eid | emp.name | emp.salary | emp.destination | +----------+--------------+-------------+------------------+--+ | 1201 | Gopal | 45000 | Technical | | 1202 | Manisha | 45000 | Proof | | 1203 | Masthanvali | 40000 | Technical | | 1204 | Kiran | 40000 | Hr | | 1205 | Kranthi | 30000 | Op | +----------+--------------+-------------+------------------+--+

But when i did basic search in atlas UI for hive_table it shows no result.

Kindly confirm what i did above is correct, if Yes then how can i fix Atlas not able to view hive tables?

- Vijay Mishra

avatar
Super Guru

@Vijay Mishra,

This is same as disabling the atlas hive hook