Team, I am using HDP-2.6 and kerberos with hive1.2 and hive2.1 enabled. I have installed interactive hiveserver2 and it is running properly. I am facing the issue with beeline. I have spark2 installed and spark thrift server running. When I try to access from hive1.2-beeline, it works properly Beeline version 1.2.1000.2.6.0.3-8 by Apache Hive beeline> !connect jdbc:hive2://hdp-qa7-n1.example.com:10016/default;principal=hive/hdp-qa7-n1.example.com@EXAMPLE.COM Connecting to jdbc:hive2://hdp-qa7-n1.example.com:10016/default;principal=hive/hdp-qa7-n1.example.com@EXAMPLE.COM Enter username for jdbc:hive2://hdp-qa7-n1.example.com:10016/default;principal=hive/hdp-qa7-n1.example.com@EXAMPLE.COM: rahul Enter password for jdbc:hive2://hdp-qa7-n1.example.com:10016/default;principal=hive/hdp-qa7-n1.example.com@EXAMPLE.COM: *********** Connected to: Spark SQL (version 2.1.0.2.6.0.3-8) Driver: Hive JDBC (version 1.2.1000.2.6.0.3-8) Transaction isolation: TRANSACTION_REPEATABLE_READ But when I try to access from hive2.1-beeline it doesn't work. Can you please help where I am doing wrong? Beeline version 2.1.0.2.6.0.3-8 by Apache Hive beeline> !connect jdbc:hive2://hdp-qa7-n1.example.com:10016/;principal=hive/hdp-qa7-n1.example.com@EXAMPLE.COM Connecting to jdbc:hive2://hdp-qa7-n1.example.com:10016/;principal=hive/hdp-qa7-n1.example.com@EXAMPLE.COM 17/06/20 22:36:22 [main]: ERROR jdbc.HiveConnection: Error opening session org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:168) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:155) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:578) [hive-jdbc-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:188) [hive-jdbc-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) [hive-jdbc-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at java.sql.DriverManager.getConnection(DriverManager.java:664) [?:1.8.0_121] at java.sql.DriverManager.getConnection(DriverManager.java:208) [?:1.8.0_121] at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:209) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.Commands.connect(Commands.java:1509) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.Commands.connect(Commands.java:1404) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121] at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1104) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1143) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:976) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:886) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:502) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at org.apache.hive.beeline.BeeLine.main(BeeLine.java:485) [hive-beeline-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121] at org.apache.hadoop.util.RunJar.run(RunJar.java:233) [hadoop-common-2.7.3.2.6.0.3-8.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:148) [hadoop-common-2.7.3.2.6.0.3-8.jar:?] 17/06/20 22:36:23 [main]: WARN jdbc.HiveConnection: Failed to connect to hdp-qa7-n1.example.com:10016 Error: Could not open client transport with JDBC Uri: jdbc:hive2://hdp-qa7-n1.example.com:10016/;principal=hive/hdp-qa7-n1.example.com@EXAMPLE.COM: Could not establish connection to jdbc:hive2://hdp-qa7-n1.example.com:10016/;principal=hive/hdp-qa7-n1.example.com@EXAMPLE.COM: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) (state=08S01,code=0) ... View more

Hi Team, The jobs get hung in accepted queue and the AM consumes more memory and CPU without executing anything. Do we need to do more tuning in yarn? I have attached the screenshot for your reference. Thanks, Rahul ... View more

Hi Team, We are working for a client and they have 11000 users in AD. From ambari, I tried ldap sync using the command ambari-server sync-ldap --all but I get the following error as: Syncing all....................................................ERROR: Exiting with exit code 1. REASON: Caught exception running LDAP sync. [LDAP: error code 4 - Sizelimit Exceeded]; nested exception is javax.naming.SizeLimitExceededException: [LDAP: error code 4 - Sizelimit Exceeded]; Please help what property should I need to add in ambari properties file. Thanks, Rahul ... View more

Hi Team, I am using HUE-3.8.1 with HDP-2.4.2. While clicking on EXECUTE button in hive query from hue, the error throws as: Couldn't find log associated with operation handle: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=7e5d0a7a-8a48-43a7-92cc-a8e2a2dc2543] The error occurs only for one particular user. Other users are able to able to execute the same and able to view the logs. I checked in hiveserver2.log file and its showing for that particular user as HiveServer2-Handler-Pool: Thread-90]: operation.Operation (Operation.java:createOperationLog(206)) - Unable to create operation log file: /tmp/hive/operation_logs/1bae153d-d7a8-4072-996a-5056cce93fda/fcfdb6ea-34f2-4ee3-b java.io.IOException: No such file or directory But I am not able to understand how other users are able to fetch the directory but only one user is not able to find the directory I found that the directory exists and have permissions under /tmp folder ll /tmp/hive |grep operation_logs drwxr-xr-x 5 hive hadoop 4096 Dec 9 14:25 operation_logs Please help. Thanks Rahul ... View more

Hi Team, Running jps command in edge node shows "Process Information Unavailable" but when I run the same command in master node, it shows all the process clearly. Any help would be appreciated. [root@edge1 ~]# jps 20712 -- process information unavailable 23022 -- process information unavailable 21612 -- process information unavailable 20888 -- process information unavailable 22085 -- process information unavailable 22650 -- process information unavailable 19360 -- process information unavailable 22768 -- process information unavailable 22324 -- process information unavailable 29348 -- process information unavailable 20591 -- process information unavailable 19212 -- process information unavailable 24930 Main 20433 -- process information unavailable 22509 -- process information unavailable 22878 -- process information unavailable 20692 -- process information unavailable [root@edge1 ~]# java -version java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) [root@edge1 ~]# echo $JAVA_HOME /usr/java/jdk1.7.0_67 [root@edge1 ~]# which jps /usr/java/jdk1.7.0_67/bin/jps [root@mst1 ~]# jps 5799 Elasticsearch 20959 RunJar 3493 SparkSubmit 15483 JobHistoryServer 14100 AmbariServer 21783 jar 31019 Jps 4617 EmbeddedServer 8597 HMaster 2859 Main 4415 JournalNode 12748 Bootstrap 6858 NameNode 30657 UnixAuthenticationService 9303 RunJar 10945 HistoryServer 4698 QuorumPeerMain 11951 ApplicationHistoryServer 19142 RunJar 5819 DFSZKFailoverController 16870 ResourceManager 10152 HRegionServer [root@mst1 ~]# echo $JAVA_HOME /usr/java/jdk1.7.0_67 [root@mst1 ~]# which java /usr/java/jdk1.7.0_67/bin/java [root@mst1 ~]# which jps /usr/java/jdk1.7.0_67/bin/jps [root@mst1 ~]# java -version java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) ... View more

Hi Team, The region servers gets stopped automatically and it throws error. We have increased the maxClient Connections to 200 in zookeeper but still it is throwing errors. Please help. 2016-10-19 02:01:04,905 INFO [main-SendThread(host2.example.com:2181)] zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x3579f98459f001c has expired, closing socket connection 2016-10-19 02:01:04,906 FATAL [main-EventThread] regionserver.HRegionServer: ABORTING region server host1.example.com,16020,1476372293108: regionserver:16020-0x3579f98459f001c, quorum=host1.example.com:2181,host2.example.com:2181,host3.example.com:2181, baseZNode=/hbase-secure regionserver:16020-0x3579f98459f001c received expired from ZooKeeper, aborting org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:613) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:524) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:534) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) Thanks, Rahul ... View more

Hi Team, Can I configure an additional AD in kerberized cluster through ambari as I am using HDP-2.4.2 with Ambari-2.2.2 ?? The reason I am asking this is because, in case the first AD server goes down, then the hadoop services should be able to kinit with the additional AD server. Otherwise we are blocked and have to wait until the AD server is accessible. Any thoughts? Thanks, Rahul ... View more