Member since
06-10-2017
39
Posts
1
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1214 | 02-20-2020 11:15 AM | |
4436 | 06-11-2018 02:04 PM | |
3690 | 06-06-2018 05:22 PM |
12-13-2022
07:17 AM
Thanks for the responses. This is a old one and resolved long back.
... View more
11-10-2021
09:23 AM
2021-11-02 09:27:36,555 WARN org.apache.hive.common.util.RetryUtilities$ExponentiallyDecayingBatchWork: [HiveServer2-Background-Pool: Thread-180]: Exception thrown while processing using a batch size 15 org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Expecting a partition with name extract_date=2018-02-15, but metastore is returning a partition with name extract_date=2018-02-15 .) at org.apache.hadoop.hive.ql.metadata.Hive.createPartitions(Hive.java:2201) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.exec.DDLTask$1.execute(DDLTask.java:2020) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.exec.DDLTask$1.execute(DDLTask.java:1999) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hive.common.util.RetryUtilities$ExponentiallyDecayingBatchWork.run(RetryUtilities.java:93) [hive-common-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.exec.DDLTask.createPartitionsInBatches(DDLTask.java:2027) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1918) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:413) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2200) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1843) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1563) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1339) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1334) [hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:256) [hive-service-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hive.service.cli.operation.SQLOperation.access$600(SQLOperation.java:92) [hive-service-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:345) [hive-service-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_231] at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_231] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) [hadoop-common-3.0.0-cdh6.2.1.jar:?] at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:357) [hive-service-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_231] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_231] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_231] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_231] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231] Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Expecting a partition with name extract_date=2018-02-15, but metastore is returning a partition with name extract_date=2018-02-15 . at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partitions_req_result$add_partitions_req_resultStandardScheme.read(ThriftHiveMetastore.java:64399) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partitions_req_result$add_partitions_req_resultStandardScheme.read(ThriftHiveMetastore.java:64358) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_partitions_req_result.read(ThriftHiveMetastore.java:64281) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_add_partitions_req(ThriftHiveMetastore.java:1819) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_partitions_req(ThriftHiveMetastore.java:1806) ~[hive-exec-2.1.1-cdh6.2.1.jar:2.1.1-cdh6.2.1]
... View more
11-09-2021
08:13 AM
Hi, We recently upgraded the cluster from CDH 5.16.2 to CDH 6.2.1 and after the upgrade MSCK repair in hive is failing for random partitions and for random tables, does anyone has any insights on how to fix this issue? Thanks
... View more
Labels:
- Labels:
-
Apache Hive
01-07-2021
01:26 PM
Hi All, Yarn RM UI and History server are throwing java.lang.illegalargumentexception when try to launch. Java version: 1.8.0_241 CDH: 6.2.1 Any Suggestions on how to over come this error.
... View more
Labels:
- Labels:
-
Apache YARN
11-30-2020
11:47 AM
Recently we added few jars to the AUX jar location in our environments and I can see the desired functions from Hive and these functions are accessible by all users who are connecting to hadoop. Is there any way to restrict the newly added functions(added UDF jar file on all hive and HS2 servers) only to fewer users instead of all using roles?
current env: CDH 6.21.
... View more
Labels:
02-20-2020
11:15 AM
@venkatsambath Thanks for your input's on this. I built the chart and your redirect provided extra metrics.
... View more
02-20-2020
08:10 AM
Hi all,
Does any one know how to create timeseries graph for httpfs service from cloudera manager.
Currently we have httpfs service on 3 servers and We need to build a graph on top of it. Is there any query which I Can use of it to create a graph.
Thanks,
Raghu.
... View more
Labels:
- Labels:
-
Cloudera Manager
01-17-2020
06:32 AM
Thanks Rohit, I tried both options but it did not helped me. Apparently cloudera engineer is saying that this option(i.e, connecting to hive via zookeeper) will not be supported in CDH 5.16.2 version. Is it true? beeline -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;transportMode=http;httpPath=cliservice;" Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release scan complete in 1ms Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;transportMode=http;httpPath=cliservice; 20/01/17 09:27:26 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0 Error: Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 configs from ZooKeeper (state=08S01,code=0) Beeline version 1.1.0-cdh5.16.2 by Apache Hive beeline> beeline -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;" Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release scan complete in 1ms Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive; java.lang.NullPointerException Beeline version 1.1.0-cdh5.16.2 by Apache Hive 0: jdbc:hive2://ZK1:2181, (closed)> Thanks, Raghu.
... View more
01-15-2020
06:50 AM
Thanks Lewis, but it didn't worked!! Caused by: java.lang.IllegalArgumentException: A HostProvider may not be empty! at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:72) at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445) at org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29) at org.apache.curator.framework.imps.CuratorFrameworkImpl$2.newZooKeeper(CuratorFrameworkImpl.java:154) at org.apache.curator.HandleHolder$1.getZooKeeper(HandleHolder.java:94) at org.apache.curator.HandleHolder.internalClose(HandleHolder.java:128) at org.apache.curator.HandleHolder.closeAndClear(HandleHolder.java:71) at org.apache.curator.ConnectionState.close(ConnectionState.java:114) ... 30 more Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper (state=,code=0) Beeline version 1.1.0-cdh5.16.2 by Apache Hive beeline>
... View more
01-15-2020
06:15 AM
Hi,
I'm trying to connect to hive from zookeeper, I'm getting a java null pointer exception with beeline cmd. currently using CDH 5.16.2 version.
I have a valid kerberos ticket:
beeline --verbose=true -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice;" Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release issuing: !connect jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice; '' [passwd stripped] scan complete in 2ms Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice; java.lang.NullPointerException at org.apache.thrift.transport.TSocket.open(TSocket.java:209) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:203) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:168) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:146) at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:211) at org.apache.hive.beeline.Commands.connect(Commands.java:1529) at org.apache.hive.beeline.Commands.connect(Commands.java:1424) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52) at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1139) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1178) at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:818) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:898) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:518) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:226) at org.apache.hadoop.util.RunJar.main(RunJar.java:141) Beeline version 1.1.0-cdh5.16.2 by Apache Hive 0: jdbc:hive2://ZK1:2181, (closed)>
... View more
Labels: