You have reached the max number of files for one folder, and an ls on this folder may not work Maybe your process is creating too many small files, maybe worth check why this is happening. For a quick workaround you can try the following : 1#get the total count of the table
2#get the creations script, make sure the table is partitioned accordingly.
3#take a copy of the table
create table tablecopy as select * from table;
4#check the count on the new table
select count(*) from table
5#check number of hdfs files
hdfs dfs -ls /apps/hive/warehouse//table
6#take a copy of the hdfs folder for further investigation
export HADOOP_HEAPSIZE="8096"
hdfs dfs -cp /apps/hive/warehouse//table /tmp
=> you may have OutOfMemoryError: GC overhead limit exceeded
7#truncate original table
truncate table table;
8#drop table
drop table table;
9#make sure hdfs folder is removed
10#create table again
11#put the data back with insert
... View more
