Member since
01-25-2020
6
Posts
1
Kudos Received
0
Solutions
07-06-2021
10:26 AM
@diplompils It is not necessary that file is lost if you are getting the output as false for recoverLease command. Usually file can't be deleted until it has lease acquired and not explicitly deleted using rm command. You can try below- hdfs debug recoverLease -path <file> -retries 10 Or you may check - https://issues.apache.org/jira/browse/HDFS-8576
... View more
07-23-2020
07:39 AM
1 Kudo
Check for more details and got error as : "Unexpected end of input stream" Now, Get the hdfs LOCATION for the table by using below command on HUE or HIVE shell: show create table <table-name>; Check for the zero byte size files and remove them from hdfs location using below command: hdfs dfs -rm -skipTrash $(hdfs dfs -ls -R <hdfs_location> | grep -v "^d" | awk '{if ($5 == 0) print $8}') Try running again our query which ran successfully this time.
... View more
07-23-2020
07:37 AM
We got the same issue and resolve as below: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask Check for more details and got error as : "Unexpected end of input stream" Now, Get the hdfs LOCATION for the table by using below command on HUE or HIVE shell: show create table <table-name>; Check for the zero byte size files and remove them from hdfs location using below command: hdfs dfs -rm -skipTrash $(hdfs dfs -ls -R <hdfs_location> | grep -v "^d" | awk '{if ($5 == 0) print $8}') Try running again our query which ran successfully this time.
... View more
07-23-2020
07:28 AM
Run msck command for the table you want to truncate in hive shell. hive> use <database-name>; hive> msck repair table <table-name>; If it will show any error then rectify it as we got one of our partition was missing. So we create that partition directory on hdfs location and re-run msck repair command. Now it would not show any issue. Now running truncate command will run successfully. hive> truncate table <table-name>; [NOTE: Please update database and table name as per the requirement]
... View more
06-26-2020
09:20 AM
Hi, We are also getting the same error almost every week for any database. Can we get any permanent solution for this? After waiting for sometime it get resolved itself. But i want to know the root cause for this and if it database or table gets locked, how i can release that lock for those DB and table in MySQL database.
... View more