Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Dropping Hive Table Crashes Hiveserver2

Dropping Hive Table Crashes Hiveserver2

Contributor

HDP 2.3, Ambari 2.1

------------------------------

While dropping a Hive External Table using beeline - Hiveserver2, the connection between Hiveserver2 and Hive Metastore process got lost. This is happening only in case of 'DROPPING Table'. Please see the error message from Hiveserver2.log:

------------------------------------------------------------------------------------------------------------------------------------------------------------------

2016-03-18 21:02:22,974 INFO [HiveServer2-Background-Pool: Thread-98]: metastore.ObjectStore (ObjectStore.java:setConf(277)) - Initialized ObjectStore

2016-03-18 21:02:22,996 INFO [HiveServer2-Background-Pool: Thread-98]: metadata.HiveUtils (HiveUtils.java:getMetaStoreAuthorizeProviderManagers(353)) - Adding metastore authorization provider: org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider

2016-03-18 21:02:23,002 INFO [HiveServer2-Background-Pool: Thread-98]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(747)) - 3: get_table : db=css tbl=xxxx

2016-03-18 21:02:23,003 INFO [HiveServer2-Background-Pool: Thread-98]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(372)) - ugi=userabc ip=unknown-ip-addr cmd=get_table : db=css tbl=xxxx

2016-03-18 21:02:23,031 INFO [HiveServer2-Background-Pool: Thread-98]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(747)) - 3: drop_table : db=css tbl=xxxx

2016-03-18 21:02:23,032 INFO [HiveServer2-Background-Pool: Thread-98]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(372)) - ugi= userabc ip=unknown-ip-addr cmd=drop_table : db=css tbl=xxxx

2016-03-18 21:02:23,142 ERROR [HiveServer2-Background-Pool: Thread-98]: hdfs.KeyProviderCache (KeyProviderCache.java:createKeyProviderURI(87)) - Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!

2016-03-18 21:02:23,168 WARN [HiveServer2-Background-Pool: Thread-98]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:executeWithArray(1630)) - Failed to execute [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] with parameters [xxxx, css]

javax.jdo.JDODataStoreException: Error executing SQL query "select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc".

at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)

at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:321)

at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:1620)

at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilterInternal(MetaStoreDirectSql.java:466)

at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitions(MetaStoreDirectSql.java:393)

at org.apache.hadoop.hive.metastore.ObjectStore$2.getSqlResult(ObjectStore.java:1738)

at org.apache.hadoop.hive.metastore.ObjectStore$2.getSqlResult(ObjectStore.java:1734)

at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2394)

at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsInternal(ObjectStore.java:1734)

at org.apache.hadoop.hive.metastore.ObjectStore.getPartitions(ObjectStore.java:1728)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)

at com.sun.proxy.$Proxy7.getPartitions(Unknown Source)

at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.dropPartitionsAndGetLocations(HiveMetaStore.java:1691)

at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1530)

at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_with_environment_context(HiveMetaStore.java:1735)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)

at com.sun.proxy.$Proxy8.drop_table_with_environment_context(Unknown Source)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.drop_table_with_environment_context(HiveMetaStoreClient.java:2062)

2 REPLIES 2

Re: Dropping Hive Table Crashes Hiveserver2

@Sushil Saxena

It seems your table partitions are creating a problem.

Try to drop a table without partitions and data? just to test.

Highlighted

Re: Dropping Hive Table Crashes Hiveserver2

Contributor

What ever the command in Hive (drop the table with or without partition), this should not crash connection between Hiveserver2 and Metastore process.