Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Namenode is in Hung state due to deadlock error

Highlighted

Namenode is in Hung state due to deadlock error

New Contributor

Hi,

Today our active NN went down due to thread deadlock's issue. This is the sample from the thread dump. All the IPC server handlers for both 8020 and 8040 were hung due to the below issue. Could you please explain about the lock issue.

org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:422) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347) Thread 372 (IPC Server handler 83 on 8040): State: WAITING Blocked count: 20620 Waited count: 2798944 Waiting on java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@4391b320 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)

1 REPLY 1

Re: Namenode is in Hung state due to deadlock error

New Contributor

Hi,

Could anyone please help me on this asap. We are facing frequent fail overs with this issue.

Don't have an account?
Coming from Hortonworks? Activate your account here