@Jay actually it should be like this hadoop fs -rm -r -skipTrash hdfs://hdfsha/tmp/hive/hive/* need to add the "*" after slash in order to delete only the folders under /tmp/hive/hive and not the sub folder itself (/tmp/hive/hive) ... View more

@Jay , in my cluster ( HDFS --> config ) I see dfs.nameservices=hdfsha , so it should be like this? hadoop fs -rm -r -skipTrash hdfs://hdfsha/tmp/hive/hive/ ... View more

@jay - nice I see there the option: hadoop fs -rm -r -skipTrash hdfs://mycluster/tmp/hive/hive/ this option will remove all folders under /tmp/hive/hive but what is the value - mycluster ? ( what I need to replace instead that ? ... View more

@jay - nice I see there the option: hadoop fs -rm -r -skipTrash hdfs://mycluster/tmp/hive/hive/ this option will remove all folders under /tmp/hive/hive but what is the value - mycluster ? ( what I need to replace instead that ? ... View more

and we have folder under /tmp/hive/hive so why cli return - Cannot find any scratch directory to clear [hdfs@master01 hive]$ hdfs dfs -ls /tmp/hive/hive Found 4 items drwx------   - hive hdfs          0 2019-06-16 21:58 /tmp/hive/hive/2f95f6a5-76ad-487e-968c-1873264a3a9c drwx------   - hive hdfs          0 2019-06-16 21:45 /tmp/hive/hive/368d201c-cedf-48dc-bbad-f13d6aed7016 drwx------   - hive hdfs          0 2019-06-16 21:58 /tmp/hive/hive/717fb013-535b-4279-a12e-4fc4261c4d68 drwx------   - hive hdfs          0 2019-06-16 21:46 /tmp/hive/hive/a58a19fe-2fc1-4b71-82ec-3307de8e2d56 ... View more

@Jay when I run it on test lab we get why ? [hive@master01 hive]$ hive --service cleardanglingscratchdir Cannot find any scratch directory to clear ... View more

@dear Jay , ok our hive version is lower so we need to run the - ( from hive user ) do you think this cli will able to remove all old folders under /tmp/hive/hive ? hive --service cleardanglingscratchdir ... View more

@Dear KJay so finally lets summary when we set the following hive.server2.clear.dangling.scratchdir=true hive.start.cleanup.scratchdir=true and then we restart the hive service from ambari do you think this configuration will be able to delete the old folders under /tmp/hive/hive in spite the folder are a millions folders ? ... View more

Dear Geoffrey Shelton Okot , since this is production cluster we need to get approve to restart the HDFS and Yarn and map reduce ( as you know this settings require to restart all 3 services ) ... View more

@ear Jay so finally lets summary when we set the following hive.server2.clear.dangling.scratchdir=true hive.start.cleanup.scratchdir=true and then we restart the hive service from ambari do you think this configuration will be able to delete the old folders under /tmp/hive/hive in spite the folder are a millions folders ? ... View more