07-27-201708:22 PM - last edited on 07-28-201706:05 AM by cjervis
Hi,Now i'm use 5.11.0release,when i use hiveserver2 ,there are some question, the hive i used 8days ago,the job of use hiveserver2 become too slow, and spent too time .I find the reason， the hiverserver2‘s have much thread are in blocked status, the sum of blocked status is 12000, the next is the memory of hiveserver2 still too high.when i restart the hiveserver2 ,all of then become normal., the old version(5.7.1) don't hive this problem ,Maybe from 5.11.0 have input the bug ,cause hiveserver2 become too slow,Who can help solve the problem ,thanks !
There was a bug added, I thought it was added in CDH 5.11.1, but I may be off on that.
It was added when the community added parallelism to the input path listing process. Threads are leaked and after some time this will slow down HS2. There is a workaround that I have confirmed works. Please review the below post and subsequent JIRA.
No，The Problem Is not the same as you said， when I update 5.11.0 to 5.11.1，the problem still ，and when restart the hiveserver2 deamon,all of they become normal.but , 7days later, use jstack -l pid | grep BLOCKED |wc -l ,the sum of this also hive 10k or more.all of the sql job become slow ,especially create partition
The bug I mentioned is in CDH 5.11.1 and hasn't been fixed yet. The workaround is to change the number of threads available to the input listing process. Change this value and restart HS2. You should stop seeing such high number of threads.