Reply
New Contributor
Posts: 1
Registered: ‎07-27-2017

CDH5.11.0 have bug in hiveserver2

[ Edited ]
Hi,Now i'm use 5.11.0release,when i use hiveserver2 ,there are some question, the hive i used 8days ago,the job of use hiveserver2 become too slow, and spent too time .I find the reason, the hiverserver2‘s have much thread are in blocked status, the sum of blocked status is 12000, the next is the memory of hiveserver2 still too high.when i restart the hiveserver2 ,all of then become normal., the old version(5.7.1) don't hive this problem ,Maybe from 5.11.0 have input the bug ,cause hiveserver2 become too slow,Who can help solve the problem ,thanks !
Posts: 614
Topics: 3
Kudos: 93
Solutions: 61
Registered: ‎08-16-2016

Re: CDH5.11.0 have bug in hiveserver2

There was a bug added, I thought it was added in CDH 5.11.1, but I may be off on that.

It was added when the community added parallelism to the input path listing process. Threads are leaked and after some time this will slow down HS2. There is a workaround that I have confirmed works. Please review the below post and subsequent JIRA.

https://community.cloudera.com/t5/Batch-SQL-Apache-Hive/HiveServer2-has-more-than-10K-waiting-daemon...
New Contributor
Posts: 1
Registered: ‎07-27-2017

Re: CDH5.11.0 have bug in hiveserver2

No,The Problem Is not the same as you said, when I update 5.11.0 to 5.11.1,the problem still ,and when restart the hiveserver2 deamon,all of they become normal.but , 7days later, use  jstack -l pid | grep BLOCKED |wc -l ,the sum of this also hive 10k or more.all of the sql job become slow ,especially create partition 
Posts: 614
Topics: 3
Kudos: 93
Solutions: 61
Registered: ‎08-16-2016

Re: CDH5.11.0 have bug in hiveserver2

The bug I mentioned is in CDH 5.11.1 and hasn't been fixed yet. The workaround is to change the number of threads available to the input listing process. Change this value and restart HS2. You should stop seeing such high number of threads.

hive.exec.input.listing.max.threads = 1
Announcements