Support Questions
Find answers, ask questions, and share your expertise

HiveException: Unable to rename output from one location to another using s3a

HiveException: Unable to rename output from one location to another using s3a

Explorer

we are currently on HDP 2.6.5
lately all or mostly all of our job are failing with following error

we have also observed that with large output and large number of files probability of these error happening is increasing . is this related to something number of threads handling metadata operations ? .any pointers or help would be greatly appreciated ..thanks in advance

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: s3a://***-dp-datalake-processed-us-east-1-prod/retention/.hive-staging_hive_2019-02-15_00-29-47_320_5587721804252525679-1/_task_tmp.-ext-10000/
cd=2018-12-26/_tmp.000026_1 to: s3a://***-dp-datalake-processed-us-east-1-prod/retention/.hive-staging_hive_2019-02-15_00-29-47_320_5587721804252525679-1/_tmp.-ext-10000/cd=2018-12-26/000026_1