Hello @tusharkathpal  What i mean that i am not using an application but web link. My rstudio web server is running on linux et the client is connecting with http://IP_ADDRESS:8787/.   Thanks. BR!! ... View more

Problem is still there.   21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.0-1127.19.1.el7.x86_64 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:user.name=eagledev 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/user1 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/user1 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=hdp-slave1.mydomain.com:2181,hdp-slave2.mydomain.com:2181,hdp-master.mydomain.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@5ace1ed4 21/06/09 16:50:22 INFO zookeeper.ClientCnxn: Opening socket connection to server hdp-slave1.mydomain.com/10.200.104.188:2181. Will not attempt to authenticate using SASL (unknown error) 21/06/09 16:50:22 INFO zookeeper.ClientCnxn: Socket connection established to hdp-slave1.mydomain.com/10.200.104.188:2181, initiating session 21/06/09 16:50:22 INFO imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting 21/06/09 16:50:22 INFO zookeeper.ClientCnxn: Session establishment complete on server hdp-slave1.mydomain.com/10.200.104.188:2181, sessionid = 0x279ef5fd2c3006b, negotiated timeout = 60000 21/06/09 16:50:22 INFO zookeeper.ZooKeeper: Session: 0x279ef5fd2c3006b closed 21/06/09 16:50:22 INFO zookeeper.ClientCnxn: EventThread shut down org.apache.curator.CuratorZookeeperClient.startAdvancedTracer(Ljava/lang/String;)Lorg/apache/curator/drivers/OperationTrace; Beeline version 3.1.0.3.1.4.0-315 by Apache Hive 0: jdbc:hive2://hdp-slave1.mydomain.com:2 (closed)> ... View more

@dmharshit    Check your Ambari UI--->HDFS-->Config-- look for the parameter hadoop.proxyuser.hive.hosts make sure it's the HS2 host is listed here. In my case bern.swiss.ch   Then restart the stale services and revert if you see any errors share the logs Happy Hadooping ... View more

@tusharkathpal , got hit to another issue, working on that. Will revert with these results by Monday. ... View more

Hello @Charles25    I would like to see the HS2 logs when the connection is being initiated. Need to understand why is the HS2 rejecting connections if the connection from client is hitting HS2.   ... View more

Hello Team,   First thing, are you able to connect to HS2 from any of the edge node? If that is connecting successfully, could you share the same to ensure we form the right connection string here.   Also, could you attach the trace logs here and HS2 logs parallelly at the same as well. ... View more

Thanks @tusharkathpal . Is there a easy way to determine the cause of the failure (catalog server vs resource crunch), given that it happens fairly inconsistent in my case. ... View more

@tusharkathpal  okay got it thank you....!   So, here I need to export data from hive/impala into RDBMS table.   Can I export data into MySQL table which have already have some records, previously.. will the export delete the existing data in the MYSQL table?   We are planning to export 1 year data from impala to MySQL table. Can you please help with suggestion.   Thank You ... View more

Hello @saamurai    Thanks for the confirmation.   Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

Thanks for your reply @tusharkathpal,   We already addressed this case with our customer to have the load balancer and vIP. but the customers seeing this is a risk points to expose the worker node IPs to the external departments within the same Organization and needs to be handled internally within the same application ~ department.   Is there any alternative scenarios that can be achieve the same approach without exposing the worker node IPs to external system like F5?   ... View more

Hello @uk_travler    Compaction will not honour hive.compactor.job.queue. Basically compactions works differently for fully acid tables and insert only tables.   For fully acid tables, when you perform a manual/auto compaction, there are two jobs spawned, one MR which is responsible for compaction which will honour compaction queue and another tez job which is responsible for stats analysis and is a tez job submitted to default queue.   For inserts only tables, when you perform a manual/auto compaction, there is tez job spawned which is submitted to default queue.   There is a jira raised raised which is being worked on it. Bug details for your reference.   HIVE-24781   let me know if you have any doubts on the above. ... View more

Try to increase the heap size for nodemanager from Yarn -> Configs and see if that resolves your issue.  If not then probably you will have to do a performance or query tuning.   Please accept this answer if it helps you resolve your issue. ... View more

@anujseeker    and the HMS logs ? ... View more

hello @pphot    You can migrate HMS and HS2 instances to any other hosts. So you can add another hosts for HS2 and HMS instance and remove the previous ones once the new ones are added and are functioning normally.   For backend database, if you migrate then you have to update the configuration in the CM so that HMS has the updated information and access as to which host it needs to communicate in order to access its backend DB. Please note this is mandatory as Hive has all the information stored in its backend DB.   Let me know if the above helps.   Regards, Tushar ... View more

Hello @anujseeker    It seems you are using the wrong path of hive. Below command works for me.   hive --orcfiledump -d --rowindex 5 /warehouse/tablespace/managed/hive/tkathpal.db/orctable/delta_0000001_0000001_0000/bucket_00000   Now in my case, hive points to the actual parcel.   [root@c2511-node1 ~]# which hive /usr/bin/hive [root@c2511-node1 ~]# ls -lt /usr/bin/hive lrwxrwxrwx 1 root root 22 Aug   3   2020 /usr/bin/hive -> /etc/alternatives/hive [root@c2511-node1 ~]# ls -lt /etc/alternatives/hive lrwxrwxrwx 1 root root 62 Aug   3   2020 /etc/alternatives/hive -> /opt/cloudera/parcels/CDH-7.1.1-1.cdh7.1.1.p0.3266817/bin/hive   So when I run hive, ultimately the jars is being picked up from the right path. Could you please check the same on your end?   ... View more

I encountered the same problem. Has your problem been solved?   2020-12-17T18:09:45,274 INFO [main]: server.HiveServer2 (HiveServer2.java:stop(913)) - Shutting down HiveServer2 2020-12-17T18:09:45,274 INFO [main]: server.HiveServer2 (HiveServer2.java:stop(925)) - Web UI has stopped 2020-12-17T18:09:45,276 ERROR [main]: server.HiveServer2 (HiveServer2.java:stop(944)) - Error removing znode for this HiveServer2 instance from ZooKeeper. java.lang.NullPointerException: null at org.apache.hive.service.server.HiveServer2.removeServerInstanceFromZooKeeper(HiveServer2.java:677) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2.stop(HiveServer2.java:942) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1090) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2.access$1700(HiveServer2.java:135) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1341) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1185) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_241] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_241] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_241] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_241] at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.1.3.0.1.0-187.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.1.3.0.1.0-187.jar:?] 2020-12-17T18:09:45,276 INFO [main]: server.HiveServer2 (HiveServer2.java:stopOrDisconnectTezSessions(890)) - Stopping/Disconnecting tez sessions. 2020-12-17T18:09:45,276 WARN [main]: server.HiveServer2 (HiveServer2.java:startHiveServer2(1100)) - Error starting HiveServer2 on attempt 1, will retry in 60000ms org.apache.hive.service.ServiceException: Failed to Start HiveServer2 at org.apache.hive.service.CompositeService.start(CompositeService.java:80) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:706) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1073) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2.access$1700(HiveServer2.java:135) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1341) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1185) [hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_241] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_241] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_241] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_241] at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.1.3.0.1.0-187.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.1.3.0.1.0-187.jar:?] Caused by: java.lang.RuntimeException: Failed to init thrift server at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initServer(ThriftBinaryCLIService.java:162) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:216) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.CompositeService.start(CompositeService.java:70) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] ... 11 more Caused by: javax.security.auth.login.LoginException: Unsupported authentication type KERBEROS at org.apache.hive.service.auth.HiveAuthFactory.getAuthTransFactory(HiveAuthFactory.java:168) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initServer(ThriftBinaryCLIService.java:72) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:216) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] at org.apache.hive.service.CompositeService.start(CompositeService.java:70) ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187] ... 11 more ... View more

IIRC, The ImpalaD process runs as the "impala".   If you have sentry grants that enable access to roles the user is in any HDFL ACLs to the contrary will be ignored.   The proper way to do this is to use Sentry to grant access to the roles which map to  groups and users, you want to have access and then enable HDFS ACLs to sync the HDFS access. ... View more

Hi @tusharkathpal , thank you for the tip. I don't have Hive itself in my infra, just Impala + S3 + Kudu, but I will find time to install it and try. However please note that I am able to run other DDLs from Impala and it is able to communicate with Hive Metastore. ... View more