Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hbase PriorityRpcServer.handler Waiting for a call (since 406hrs)

Solved Go to solution
Highlighted

hbase PriorityRpcServer.handler Waiting for a call (since 406hrs)

Explorer

Hi admin users, I am using hbase cluster CM 5.7. often i check the hmaster status thru webUI. i noticed something is waiting for queue to process. does anyone help me understand what this means. what is this waiting for??

 

Master: hmaster01.abc.net

Fri Jul 29 12:55:49 CDT 2016B.defaultRpcServer.handler=29,queue=2,port=60000WAITING (since 12sec ago)Waiting for a call (since 12sec ago)
Fri Jul 29 12:55:49 CDT 2016PriorityRpcServer.handler=0,queue=0,port=60000WAITING (since 406hrs, 23mins, 18sec ago)Waiting for a call (since 406hrs, 23mins, 18sec ago)
Fri Jul 29 12:55:49 CDT 2016PriorityRpcServer.handler=1,queue=1,port=60000WAITING (since 406hrs, 23mins, 18sec ago)Waiting for a call (since 406hrs, 23mins, 18sec ago)
 
 
1 ACCEPTED SOLUTION

Accepted Solutions

Re: hbase PriorityRpcServer.handler Waiting for a call (since 406hrs)

Master Guru
The cause for crashes would be unrelated to this observance.

I'd recommend starting a new topic by posting the logs of the service that crashes for you, specifically the earliest FATAL message it produces before it aborts, if there is one.
3 REPLIES 3

Re: hbase PriorityRpcServer.handler Waiting for a call (since 406hrs)

Master Guru
The HMaster has two RPC handler pools, one for priority tasks, the other for regular ones. The regular ones see activity regularly and are represented by the section "B.defaultRpcServer" and you'll find that these continually process client or RS calls made to them.

The priority ones, on the other hand, only see limited activity relating to certain specific work, and will mostly be idle depending on your cluster's situation. It is normal to see a very large "Waiting since" value on the priority handlers, as they are used only in special situations.

Some more bits for your information:
- "Waiting for a call" means that the thread is idle and is waiting for some work to flow in.
- The "queue=X" value here is a queue identifier associated to the specific handler thread ID (there are multiple parallel queues that the handlers feed themselves from) and not the size of queue.

These waiting statuses on the HMaster UI only appear if you switch the Tasks tabs to show all monitored tasks or all RPC tasks regardless of state. If you're interested in finding actual calls, use the "Show Active RPC Calls" tab under it instead, and you may be able to catch some of the calls that regularly come in.

Re: hbase PriorityRpcServer.handler Waiting for a call (since 406hrs)

Explorer

so it is mostly like info kind message we dont need to worry. currently my cluster is occuring frequent crashes, so this post. 

 

Thanks for clarification!!

Re: hbase PriorityRpcServer.handler Waiting for a call (since 406hrs)

Master Guru
The cause for crashes would be unrelated to this observance.

I'd recommend starting a new topic by posting the logs of the service that crashes for you, specifically the earliest FATAL message it produces before it aborts, if there is one.
Don't have an account?
Coming from Hortonworks? Activate your account here