Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Data gets vanished in HDFS after Move fail in Hive

Data gets vanished in HDFS after Move fail in Hive

Champion

I have HiveServer2 , using beeline . Configured Shared Hive metastore for  hive / impala . 

I was just noticing this strange behaviour when building a fresh enviroment.  I missed permission for hive / impala on the hdfs directory level , we are not using sentry at this point. Used sqoop  import to move the Table data to hdfs.

When tried to Load Statement in Hive after creating Table . it fail and it complained about permission 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

 I am able to fix the issue . but the real problem is the data in the hdfs was vanished . when i perfomed hadoop fs -ls there was not files just the table directory sitting . Shouldnt there be a rollback  statement in the transcation when it tried to move the file and when failed . 

Could anyone please clarify this . Also is there anyway we could recover the data 

1 REPLY 1
Highlighted

Re: Data gets vanished in HDFS after Move fail in Hive

Guru

Hi,

 

Couple of things that I need to look into further:

 

- what was the FULL query you ran in Hive?

- what was the FULL ERROR message reported in beeline?

- what was the FULL ERROR message in HS2 log?

 

I would like to see at what stage the error happened.

 

Thanks

Don't have an account?
Coming from Hortonworks? Activate your account here