When I have a table at my sqoop schema and want to delete it, I go to HIVE editor (through HUE) and key in the following command DROP TABLE IF EXISTS schemaName.tblName PURGE;
After that the table disappeared form the gui of HUE (sqoop table list, metastore list) but the actual files of the table were not deleted from the HDFS.
From the other hand when I do the same thing at the impala editor everything works fine.
Why this is happened & how can I drop/remove a table from hive metastore and HDFS all at once?
Thanks in advance.
@BildervicI’m afraid it is not clear what I am saying. I have a table which is owned to impala user. When I am dropping the table through impala editor (Hue) with the above command the table & hdfs files were deleted.
But when I have a table which is owned to hive user and I am trying to delete it through hive editor (HUE) with the above command the table is disappeared from the gui but the files were not deleted.
And I am asking why is this happened ?
it looks like a permission issue which is silently ignored. Can you please post the ACL's from the HDFS path, the root folder where the table is stored and the ACL's of the table path as well.
Maybe you can check the hive server's log, if there is any kind of permission issue.
And finally I would check the NameNode logs, of the file is not deleted because of missing permissions, it will be there as a log message.
@Tomas79 here is the ACL of the hdfs path of the dir/table.
# file: /user/hive/warehouse/sqoop.db/sr_tt
# owner: hive
# group: hive
# flags: --t
Now if I go from Hive editor (HUE) and execute the query DROP TABLE sr_tt PURGE;
the files were not deleted.
Also Do you know where can I found the logs of the NameNode and hive server's log?
@eMazarakis I noticed the "-t" flag on the directory. That is a sticky bit, every time you see it in the HDFS ACLs, it tells you that only the owner is able to drop directory, nobody else (even if write access is granted)
So in your case only the hive user can remove this directory. Maybe as @EricL pointed out, you have impersonation, so the query is running under a different user.
In either way, you need to search for this permission issue in the logs.
@EricL My findings are the following: