About EricL

EricL · ‎05-08-2019

Hi, In that case, another thing I can think of is how often you update this JAR file and how often you see the error? Not sure if it could be the case that the JAR was replaced/updated while the reloading happened, hence the corrupted file (reading in the middle of updating) might be causing some classes not being loaded properly?

EricL · ‎05-08-2019

My pleasure!

EricL · ‎05-07-2019

Hi Vibin, I can see that this issue is tracked under HIVE-6980 on the upstream JIRA. And I can confirm that it has been fixed in CDH6.1 and CDH5.16 onwards. If you are using older version, you can try to increase the value for hive.metastore.client.socket.timeout as a workaround as mentioned in the previous post. Hope above is helpful. Cheers Eric

EricL · ‎05-07-2019

If you change column order, it will affect existing data, so the new position of the column will match different data, that's OK for you?

EricL · ‎05-07-2019

Hi, How big is the JAR file? Since it is running a Map Join, it runs on HS2 itself, and it will copy the JAR file locally on the HS2 host, and I think it might be under /tmp, but not 100% sure. So one thing I think you should check is the disk space. If at the time the disk was full, it might cause this kind of issue. Maybe check this first before anything else. Eric

EricL · ‎05-07-2019

As Manpreet mentioned, it is better to run in Beeline and check the error message in HiveServer2 log, which can tell more about the story. Eric

EricL · ‎05-07-2019

I don't see super easy way, but below might help: 1. if this table is not external, make it external first: ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE') 2. drop the table 3. create a new table and define same partition column 4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly 5. remove the empty directories under the table 6. msck repair table to recover the 6 current partitions Hope above can help. Eric

EricL · ‎05-07-2019

I think the question was about Partitions, not Files?

EricL · ‎05-07-2019

Hi, The easiest way I can see is to: 1. drop the table (since it is external, data should NOT be dropped) 2. create a new table on top of it and specify as partitioned by ColumnA of type timestamp (the column name should remain the same as before, can't be changed to ColumnB, otherwise step 3 will not be able to pick it up) 3. run "msck repair table {tablename}" to recover the partitions This assumes that the partition values will remain unchanged. Hope this can help.

EricL · ‎05-07-2019

Hi, When you are saying "I set all timeout parameters to 0,", can you please confirm what timeouts you set? From client side or Hive server side? Have you tried to increase the SocketTimeout on the ODBC driver side: https://www.cloudera.com/documentation/other/connectors/hive-odbc/latest/Cloudera-ODBC-Driver-for-Apache-Hive-Install-Guide.pdf on page 81.

Online	Offline
Last Visited	‎08-12-2020 03:17 AM

Member Since	‎03-23-2015 01:24 PM
Last Visited	‎08-12-2020 03:17 AM
Posts	1,288
Kudos received	113

Cloudera Community

Re: max() function generating an error in sqoop

Re: Add a dynamic variable to a Hive view

Re: Hive Server 2 failing to start CDP ,Cloudera M...

Re: Sqoop export from hive to teradata - > issue ...

Re: Cloudera Hadoop internal workings

Re: Hive reloadable udf: random 'Unable to find c...

Re: Can not ALTER or DROP a big partitionned table...

Re: Can not ALTER or DROP a big partitionned table...

Re: How can we change the column order in Hive tab...

Re: Hive reloadable udf: random 'Unable to find c...

Re: Unable to create table in hive : FAILED: Execu...

Re: Hive Partition

Re: Hive Partition

Re: How to change partition on existing Hive Table...

Re: HIVE ODBC Driver Timeout on INSERT