Skip to main content
dmharshit
dmharshit
Silver ⭐️⭐️⭐️
April 26, 2021
Question

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.StatsTask (state=08S01,code=1)

Error message:
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.StatsTask (state=08S01,code=1)

 

Following is Details of our cluster

 

HDP Version: 3.1.4
Hive Version: 3.1.0.3.1.4.0-315

 

Configs for Dynamic Partition in hive:

hive.exec.dynamic.partition=true

Allow all partitions to be Dynamic=nonstrict

hive.exec.max.dynamic.partitions=50000

hive.exec.max.dynamic.partitions.pernode=20000

Sort Partitions Dynamically=false
Allow dynamic numbers of reducers=true
Allow dynamic partition pruning=true


Any help appreciated.

 

9 replies

shobikas
Gold ⭐️
shobikas
Gold ⭐️
April 27, 2021

@dmharshit  It would be really difficult to answer for the question without the necessary details like what is the query you are running, From when you started facing the issue and the complete error stack. But from the two line which you provided i suspect that there might be issue with stats. So you can try setting the below parameter and try it.

set hive.stats.column.autogather=false;
set hive.optimize.sort.dynamic.partition=true;

If the above doesn't help kindly provide the necessary details to figure out the issue.

If the above parameter resolves the issue kindly mark "Accept as a Solution"

Thanks

Shobika S 

Translate
dmharshit
dmharshitAuthor
Silver ⭐️⭐️⭐️
April 28, 2021

Hi @shobikas, 

Thanks for your response.

I tried running queries using the configs you mentioned. Queries are getting executed successfully but no change is observed in table data.

Following queries I'm running in hive (​for mix partitioning which includes static and dynamic partitioning together)
1. insert into table foodmart.idNameGenderPartDeptTableName partition (manager_id = 200, manager_name, manager_gender) VALUES (1, 'COMP', 'John', 'M');
2. insert into table foodmart.idNameGenderPartDeptTableName partition (manager_id = 1111, manager_name, manager_gender) select department_id, full_name, first_name, gender from foodmart.trimmed_employee;
3. insert OVERWRITE table foodmart.idNameGenderPartDeptTableName partition (manager_id = 1234, manager_name, manager_gender) select department_id, full_name, first_name, gender from foodmart.trimmed_employee;
4. insert OVERWRITE table foodmart.idNameGenderPartDeptTableName partition (manager_id = 1234, manager_name, manager_gender) values (02, 'HR', 'Niks', 'M');

Translate
vidanimegh
vidanimegh
Gold ⭐️
April 29, 2021

Hi @dmharshit ,

 

Can you share the log snippets from Hive Metastore, Hiveserver2 and YARN ResourceManager from the timeframe of execution of this query?

 

Thanks,

Megh

Translate
Terms & ConditionsAccessibility statement