Hive : insert into org.apache.hadoop.hive.ql.exec.MoveTask

[#1]

Hi ,

Its a automated job (running through talend ) and its a common table where we will be performing insertions ,

sometimes it works fine and sometimes we get org.apache.hadoop.hive.ql.exec.MoveTask

We are not able to figure it out the exact reason . Please help us .

Table structure :

CREATE EXTERNAL TABLE `lz_control_zyme_file_load_tracker`(
  `target_stage_name` string,
  `input_file_sequential_number` int,
  `source_system_file_type_name` string,
  `region_code` string,
  `source_file_name` string,
  `process_date` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '|'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://HDPPRODNN/sales/channel/loading/lz_control_zyme_file_load_tracker/data'
TBLPROPERTIES (
  'COLUMN_STATS_ACCURATE'='true',
  'numFiles'='43739',
  'numRows'='4706',
  'rawDataSize'='443398',
  'totalSize'='1842648',
  'transient_lastDdlTime'='1520496190')

[#2]

@Satish Anjaneyappa

This error occurs when map-reduce/ tez job is completed and then data is moved from staging directory to the destination directory.

Error while moving could occur due to permission issues or if there is space issue in case of huge table.

Can you check the permissions or if you have space quota set at the directory level.

[#3]

@Satish Anjaneyappa

You would get more details in hive logs, suggesting why the move operation failed.

[#4]

@rtrivedi : Thanks

But

1) The same job will run successful when u re-run the job (so there ll be no space and permission issues )

2) Its a small table

[#5]

Did you find any details in logs ?

[#6]

@rtrivedi

Yes got some hive logs ..

2018-03-06 03:42:42,951 INFO  [HiveServer2-Background-Pool: Thread-1132047]: exec.MoveTask (MoveTask.java:execute(309)) - Partition is: {region_code=null, fiscal_year_week_code=null}
        at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:298)
2018-03-06 03:49:24,600 ERROR [HiveServer2-Background-Pool: Thread-1132295]: ql.Driver (SessionState.java:printError(962)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask