Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive : insert into org.apache.hadoop.hive.ql.exec.MoveTask

avatar
Contributor

Hi ,

Its a automated job (running through talend ) and its a common table where we will be performing insertions ,

sometimes it works fine and sometimes we get org.apache.hadoop.hive.ql.exec.MoveTask

We are not able to figure it out the exact reason . Please help us .

Table structure :

CREATE EXTERNAL TABLE `lz_control_zyme_file_load_tracker`(
  `target_stage_name` string,
  `input_file_sequential_number` int,
  `source_system_file_type_name` string,
  `region_code` string,
  `source_file_name` string,
  `process_date` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '|'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://HDPPRODNN/sales/channel/loading/lz_control_zyme_file_load_tracker/data'
TBLPROPERTIES (
  'COLUMN_STATS_ACCURATE'='true',
  'numFiles'='43739',
  'numRows'='4706',
  'rawDataSize'='443398',
  'totalSize'='1842648',
  'transient_lastDdlTime'='1520496190')


5 REPLIES 5

avatar
Expert Contributor

@Satish Anjaneyappa

This error occurs when map-reduce/ tez job is completed and then data is moved from staging directory to the destination directory.

Error while moving could occur due to permission issues or if there is space issue in case of huge table.

Can you check the permissions or if you have space quota set at the directory level.

avatar
Expert Contributor

@Satish Anjaneyappa

You would get more details in hive logs, suggesting why the move operation failed.

avatar
Contributor

@rtrivedi : Thanks

But

1) The same job will run successful when u re-run the job (so there ll be no space and permission issues )

2) Its a small table

avatar
Expert Contributor

Did you find any details in logs ?

avatar
Contributor

@rtrivedi

Yes got some hive logs ..

2018-03-06 03:42:42,951 INFO  [HiveServer2-Background-Pool: Thread-1132047]: exec.MoveTask (MoveTask.java:execute(309)) - Partition is: {region_code=null, fiscal_year_week_code=null}
        at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:298)
2018-03-06 03:49:24,600 ERROR [HiveServer2-Background-Pool: Thread-1132295]: ql.Driver (SessionState.java:printError(962)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask