07-24-2017 07:15 PM
Several times we have done this simulation in our testing enviroment ,its kind of wired we never faced suck problem . Just curious to know if you have HS2 configured what about ur database for metastore , is it a shared metastore for impala ? I would inspect the fsimage and edits log file using offline imagine viewer . dont alter unless you are confident .Just to see if its corrupt fsimage / edits
07-25-2017 01:06 AM
Doing a hard restart will not cause data loss, but may I ask why made you do a hard restart in the first place? This might shed some light as to what has gone wrong.
What do the contents of /user/hive/warehouse look like? If there are no files there then either your database files have been deleted or your tables are stored elsewhere. Try running "show create table table_name" to see where the files are located for one of your tables. If there are no data files in this location then they may have been moved or deleted.
Check your namenode logs to see if there were any actions that might have resulted in this happening. It could be someone has moved the warehouse directory by mistake, in which case the files won't match the Hive metadata.
07-25-2017 07:33 AM
I was having some problem with hue server, so i restarted the cluster which seems to have fixed the problem with hue. But after the restart i dont see the /user/hive/warehouse directory anymore. I can see in hdfs charts that the disk used has gone down in a straight line after the restart from about 500 GB to around 50.
I have gone through hdfs-audit and namenode logs but i didnt find anything there. And the location for data storage of hive tables is defaut i.e /user/hive/warehouse.
08-02-2017 02:50 AM