Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Hive : insert into org.apache.hadoop.hive.ql.exec.MoveTask

Explorer

Hi ,

Its a automated job (running through talend ) and its a common table where we will be performing insertions ,

sometimes it works fine and sometimes we get org.apache.hadoop.hive.ql.exec.MoveTask

We are not able to figure it out the exact reason . Please help us .

Table structure :

CREATE EXTERNAL TABLE `lz_control_zyme_file_load_tracker`(
  `target_stage_name` string,
  `input_file_sequential_number` int,
  `source_system_file_type_name` string,
  `region_code` string,
  `source_file_name` string,
  `process_date` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '|'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://HDPPRODNN/sales/channel/loading/lz_control_zyme_file_load_tracker/data'
TBLPROPERTIES (
  'COLUMN_STATS_ACCURATE'='true',
  'numFiles'='43739',
  'numRows'='4706',
  'rawDataSize'='443398',
  'totalSize'='1842648',
  'transient_lastDdlTime'='1520496190')


5 REPLIES 5

Rising Star

@Satish Anjaneyappa

This error occurs when map-reduce/ tez job is completed and then data is moved from staging directory to the destination directory.

Error while moving could occur due to permission issues or if there is space issue in case of huge table.

Can you check the permissions or if you have space quota set at the directory level.

Rising Star

@Satish Anjaneyappa

You would get more details in hive logs, suggesting why the move operation failed.

Explorer

@rtrivedi : Thanks

But

1) The same job will run successful when u re-run the job (so there ll be no space and permission issues )

2) Its a small table

Rising Star

Did you find any details in logs ?

Explorer

@rtrivedi

Yes got some hive logs ..

2018-03-06 03:42:42,951 INFO  [HiveServer2-Background-Pool: Thread-1132047]: exec.MoveTask (MoveTask.java:execute(309)) - Partition is: {region_code=null, fiscal_year_week_code=null}
        at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:298)
2018-03-06 03:49:24,600 ERROR [HiveServer2-Background-Pool: Thread-1132295]: ql.Driver (SessionState.java:printError(962)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.