Support Questions
Find answers, ask questions, and share your expertise

NameNode is getting down quite frequently

NameNode is getting down quite frequently

New Contributor

Our NameNode is getting down quite frequently. Upon checking logs, found below exception.

ERROR hdfs.DFSClient (DFSClient.java:closeAllFilesBeingWritten(959)) - Failed to close inode 4290642291 java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:478) at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) at java.io.DataOutputStream.write(DataOutputStream.java:107) at org.apache.hadoop.hdfs.DFSPacket.writeTo(DFSPacket.java:176) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:623) 2019-02-11 17:14:33,789 INFO namenode.NameNode (LogAdapter.java:info(47)) - SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at xx-xxxx-xxxx.xxxx.xxx/xx.xxx.xxx.xx ************************************************************/ hadoop-hdfs-namenode-xx-xxxx-xxxx.log lines 1009951-1009982/1009982 (END) packet_write_wait: Connection to xx.xxx.xxx.xx port 22: Broken pipe

Would be helpful on any suggestions.

Thanks!