Reply
New Contributor
Posts: 1
Registered: ‎03-15-2015

Hadoop namenode port getting blocked

I have installed 7 VM instances of Ubuntu 14.04 LTS servers.  I installed latest Cloudera manager with core Hadoop package successfully . First instance runs the namenode service and all other 6 nodes run datanode service.I think my NameNode is getting crashed or blocked due to some issue.
After rebooting if I check JPS command output my namenode is running. In core-site.xml the fs.defaultfs property is set to hdfs://instance-1:8020.
but in the netstat -tulpn output 8020 port is not there.

this is the JPS output right after rebooting.

root@instance-1:~# jps
3017 VersionInfo
2613 NameNode
3371 VersionInfo
3313 ResourceManager
3015 Main
2524 QuorumPeerMain
2877 HeadlampServer
1556 Main
3480 Jps
2517 SecondaryNameNode
3171 JobHistoryServer
2790 EventCatcherService
2842 AlertPublisher
2600 Bootstrap
2909 Main

 

this is the netstat output that I checked after jps.

 

root@instance-1:~# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 600/rpcbind 
tcp 0 0 0.0.0.0:9010 0.0.0.0:* LISTEN 2524/java 
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1164/sshd 
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN 1158/postgres 
tcp 0 0 127.0.0.1:19001 0.0.0.0:* LISTEN 1496/python 
tcp 0 0 0.0.0.0:42043 0.0.0.0:* LISTEN 2524/java 
tcp 0 0 10.240.71.132:9000 0.0.0.0:* LISTEN 1419/python 
tcp 0 0 0.0.0.0:7432 0.0.0.0:* LISTEN 1405/postgres 
tcp6 0 0 :::111 :::* LISTEN 600/rpcbind 
tcp6 0 0 :::22 :::* LISTEN 1164/sshd 
tcp6 0 0 :::7432 :::* LISTEN 1405/postgres 
udp 0 0 0.0.0.0:68 0.0.0.0:* 684/dhclient 
udp 0 0 0.0.0.0:111 0.0.0.0:* 600/rpcbind 
udp 0 0 10.240.71.132:123 0.0.0.0:* 3323/ntpd 
udp 0 0 127.0.0.1:123 0.0.0.0:* 3323/ntpd 
udp 0 0 0.0.0.0:123 0.0.0.0:* 3323/ntpd 
udp 0 0 0.0.0.0:721 0.0.0.0:* 600/rpcbind 
udp 0 0 0.0.0.0:29611 0.0.0.0:* 684/dhclient 
udp6 0 0 :::111 :::* 600/rpcbind 
udp6 0 0 :::123 :::* 3323/ntpd 
udp6 0 0 :::721 :::* 600/rpcbind 
udp6 0 0 :::22577 :::* 684/dhclient

 


As I said I don't see 8020 port. After one minute I checked JPS output and the namenode is gone.

this is the jps output one minute after rebooting.

 

root@instance-1:~# jps
3794 Main
3313 ResourceManager
3907 EventCatcherService
4325 Jps
2530 RunJar
3082 RunJar
2524 QuorumPeerMain
2656 Bootstrap
2877 HeadlampServer
1556 Main
2517 SecondaryNameNode
3171 JobHistoryServer
2842 AlertPublisher
2600 Bootstrap

 

 

As I said namenode is not there. I repeated the above process couple of times and everytime I get the same results port 8020 not there and namenode getting crashed. Please help me I am stuck here for over two days. Thanks in advance.

Posts: 1,885
Kudos: 422
Solutions: 298
Registered: ‎07-31-2013

Re: Hadoop namenode port getting blocked

Have you looked at the NAMENODE log under /var/log/hadoop-hdfs/ directory? It will tell you why its been crashing.
Announcements