have u tried move out (or delete the folder for that partition ) from hdfs, then run: msck repair table <tablename> ... View more

@peeyush Thanks for your help. Followed the same and issue been resolved. ... View more

Yes, the audit log will serve the purpose. Note that, in some cases, it is not straightforward to search the log for deletion since a directory (or a file) may not be deleted directly -- it may be deleted as a part of the deletion of its parent/ancestor directory. So we should first search the full path in the log. If it is not found, search the parent directory path and so on. It will be more complicated if deletion and re-creation occurred repeatedly. For example 1) user A: create /foo 2) user A: create /foo/bar 3) user A: del /foo 4) user B: create /foo 5) user B: del /foo Who has deleted /foo/bar? It is easy to mistakenly take user B as the answer. B is the last user deleted foo but B is not the user deleted /foo/bar. In such case, we should first determine when the target directory/file is created and then search what happened of it starting from the creation time. You can imagine that it is even harder to find out the correct answer if the path or the parent/ancestor paths are moved/renamed. We need to pay extra attention if the rename operation is involved. ... View more

@Sankar T - Can you please accept my answer if it helped! ... View more

@clukasik - Thanks for the reply. I tried your query and it worked , here is my observations. show partitions tablename; partition_dt=2017-01-03 10%3A20%3A13 partition_dt=2017-01-03 10%3A20%3A14 partition_dt=2017-01-03 10%3A20%3A15 select patition_dt, count(1) from tablename group by partition_dt; 2017-01-03 10:20:13.0 | 2191 2017-01-03 10:20:14.0 | 7888 2017-01-03 10:20:15.0 | 1647 Note: %3A should be replaced with : while quering. select count(1) from tablename where partition_dt="2017-01-03 10:20:13"; +-------+--+ | _c0 | +-------+--+ | 2191 | +-------+--+ 1 row selected (3.643 seconds) select count(1) from tablename where partition_dt="2017-01-03 10:20:14"; +-------+--+ | _c0 | +-------+--+ | 7888 | +-------+--+ 1 row selected (0.101 seconds) select count(1) from tablename where partition_dt="2017-01-03 10:20:15"; +-------+--+ | _c0 | +-------+--+ | 1647 | +-------+--+ 1 row selected (0.101 seconds) Sampling i used below query. select * from tablename where partition_dt="2017-01-03 10:20:15" limit 5; ... View more

@Sankar T Please read Wangda's blog if you are planning to setup yarn node labels - https://wangda.live/2016/04/16/suggestions-about-how-to-better-use-yarn-node-label/ ... View more

@Sagar Shimpi : will this process cleanup physical folders and dependencies automatically by running curl command or Ambari service deletion ? if not , do we need to delete manually ? thanks in advance. ... View more