@Aditya, Kindly suggest how to fix it as unable to proceed. I have reverted the changes. it started throwing the errors again. - Vijay Mishra ... View more

@Aditya I explored few more thing. Updated below property as mentioned below and restarted hive and atlas. From: hive.exec.post.hooks-org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook To: hive.exec.post.hooks-org.apache.hadoop.hive.ql.hooks.ATSHook All commands in beeline started getting executed but in Atlas UI i am unable to see hive table. [ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; +-----------+--+ | tab_name | +-----------+--+ | emp | +-----------+--+ 1 row selected (2.232 seconds) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> select * from emp; +----------+-----------+-------------+------------------+--+ | emp.eid | emp.name | emp.salary | emp.destination | +----------+-----------+-------------+------------------+--+ +----------+-----------+-------------+------------------+--+ No rows selected (1.519 seconds) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> LOAD DATA INPATH 'hdfs://hdpdev/user/ajay/sample.txt' INTO TABLE emp; 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> select * from emp; +----------+--------------+-------------+------------------+--+ | emp.eid | emp.name | emp.salary | emp.destination | +----------+--------------+-------------+------------------+--+ | 1201 | Gopal | 45000 | Technical | | 1202 | Manisha | 45000 | Proof | | 1203 | Masthanvali | 40000 | Technical | | 1204 | Kiran | 40000 | Hr | | 1205 | Kranthi | 30000 | Op | +----------+--------------+-------------+------------------+--+ But when i did basic search in atlas UI for hive_table it shows no result. Kindly confirm what i did above is correct, if Yes then how can i fix Atlas not able to view hive tables? - Vijay Mishra ... View more

@Aditya, Atlas client installed on all 4 nodes of cluster. Still i ran the command provided by you. Below snap for the same. [root@vijayhdf-1 hive]# curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"HostRoles": {"state": "INSTALLED"}}' http://vijayhdf-1.novalocal:8080/api/v1/clusters/hdpdev/hosts/vijayhdf-1.novalocal/host_components/ATLAS_CLIENT HTTP/1.1 202 Accepted X-Frame-Options: DENY X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff Cache-Control: no-store Pragma: no-cache Set-Cookie: AMBARISESSIONID=n8czyjmtzvhzevwgenc583h5;Path=/;HttpOnly Expires: Thu, 01 Jan 1970 00:00:00 GMT User: admin Content-Type: text/plain Vary: Accept-Encoding, User-Agent Content-Length: 149 { "href" : "http://vijayhdf-1.novalocal:8080/api/v1/clusters/hdpdev/requests/122", "Requests" : { "id" : 122, "status" : "Accepted" } }[root@vijayhdf-1 hive]# I restarted Hive and atlas and ran the beeline but it throws error again. [ajay@vijayhdf-1 ~]$ hostname -f vijayhdf-1.novalocal [ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; Error: Error while processing statement: FAILED: Hive Internal Error: java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook) (state=08S01,code=12) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> - Vijay Mishra ... View more

@Aditya, For verify purpose i just disable the atlas hook from hive and ran the beeline and its working. [ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; +-----------+--+ | tab_name | +-----------+--+ +-----------+--+ No rows selected (2.28 seconds) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> Post which i reenable atlas hook into hive and ran the beeline again and got the same error. [ajay@vijayhdf-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ajay Enter password for jdbc:hive2://vijayblue-1.novalocal:2181,vijayhdf-3.novalocal:2181,vijayhdf-2.novalocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******** Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> show tables; Error: Error while processing statement: FAILED: Hive Internal Error: java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook) (state=08S01,code=12) 0: jdbc:hive2://vijayblue-1.novalocal:2181,vi> ... View more

@Aditya, [root@vijayhdf-2 hive]# ll /usr/bin/beeline lrwxrwxrwx 1 root root 40 Jan 30 08:47 /usr/bin/beeline -> /usr/hdp/current/hive-client/bin/beeline [root@vijayhdf-2 hive]# cat /usr/hdp/current/hive-client/bin/beeline #!/bin/bash if [ -d "/usr/hdp/2.6.3.0-235/atlas/hook/hive" ]; then if [ -z "${HADOOP_CLASSPATH}" ]; then export HADOOP_CLASSPATH=/usr/hdp/2.6.3.0-235/atlas/hook/hive/* else export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:/usr/hdp/2.6.3.0-235/atlas/hook/hive/* fi fi BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default} [ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hbase ] && . ${BIGTOP_DEFAULTS_DIR}/hbase export HIVE_HOME=${HIVE_HOME:-/usr/hdp/2.6.3.0-235/hive} export HADOOP_HOME=${HADOOP_HOME:-/usr/hdp/2.6.3.0-235/hadoop} export ATLAS_HOME=${ATLAS_HOME:-/usr/hdp/2.6.3.0-235/atlas} HCATALOG_JAR_PATH=/usr/hdp/2.6.3.0-235/hive-hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1000.2.6.3.0-235.jar:/usr/hdp/2.6.3.0-235/hive-hcatalog/share/hcatalog/hive-hcatalog-server-extensions-1.2.1000.2.6.3.0-235.jar:/usr/hdp/2.6.3.0-235/hive-hcatalog/share/webhcat/java-client/hive-webhcat-java-client-1.2.1000.2.6.3.0-235.jar if [ -z "${HADOOP_CLASSPATH}" ]; then export HADOOP_CLASSPATH=${HCATALOG_JAR_PATH} else export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:${HCATALOG_JAR_PATH} fi exec "${HIVE_HOME}/bin/beeline.distro" "$@" [root@vijayhdf-2 hive]# ... View more

atlas-hook-hive.png @ Aditya Its enabled, Attached screenshot for the same. - Vijay Mishra ... View more

Krishna Pandey Yes, Issue was with authorization and its fixed now. - Vijay Mishra ... View more

Did some troubleshooting and found issue default topology, knox unable to activate default topology. I have corrected the issue and 404 error gone. But started getting new error i.e. 403 [ajay@vijayhdp-1 ~]$ curl -iku ajay:Passw0rd -X GET 'https://vijayhdp-1.novalocal:8443/gateway/default/webhdfs/v1/user/ajay?op=LISTSTATUS' HTTP/1.1 403 Forbidden Date: Wed, 03 Jan 2018 07:13:48 GMT Set-Cookie: JSESSIONID=1hgp22be0fp121chqv5fmgei6q;Path=/gateway/default;Secure;HttpOnly Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; Expires=Tue, 02-Jan-2018 07:13:48 GMT Content-Type: text/html; charset=ISO-8859-1 Cache-Control: must-revalidate,no-cache,no-store Content-Length: 320 Server: Jetty(9.2.15.v20160210) <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> <title>Error 403 Forbidden</title> </head> <body><h2>HTTP ERROR 403</h2> <p>Problem accessing /gateway/default/webhdfs/v1/user/ajay. Reason: <pre> Forbidden</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/> </body> </html> [ajay@vijayhdp-1 ~]$ Knox gateway log: 2018-01-03 12:43:48,296 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(724)) - Computed userDn: CN=ajay,OU=HADOOP,DC=HADOOP,DC=LOCAL using ldapSearch for principal: ajay 2018-01-03 12:43:48,506 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(724)) - Computed userDn: CN=ajay,OU=HADOOP,DC=HADOOP,DC=LOCAL using ldapSearch for principal: ajay 2018-01-03 12:43:48,528 INFO hadoop.gateway (KnoxLdapRealm.java:rolesFor(327)) - Computed roles/groups: [app] for principal: ajay Kindly suggest. - Vijay Mishra ... View more