Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Datanode won't start - dfs.datanode.http.address in use

avatar
New Contributor

One of of my data-node won't start. The error log on {HADOOP_LOG}/hdfs/{datanode}.err shown that the port of dfs.datanode.http.address already in use. We use custom port for  dfs.datanode.http.address. When I run netstat -lpn | grep 1022, no service run on it. While for {datanode}.log file is not updated.

 

 

Initializing secure datanode resources
isSaslEnabled:true
Opened streaming server at /0.0.0.0:1019
java.net.BindException: Address already in use /0.0.0.0:1022
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:461)
        at sun.nio.ch.Net.bind(Net.java:453)
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)
        at org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.getSecureResources(SecureDataNodeStarter.java:158)
        at org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.init(SecureDataNodeStarter.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:207)
Cannot load daemon
Service exit with a return value of 3

 

 

The error message on Ambari:

 

resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh  -H -E /usr/hdp/3.1.5.0-152/hadoop/bin/hdfs --config /usr/hdp/3.1.5.0-152/hadoop/conf --daemon start datanode' returned 1. ERROR: Cannot set priority of datanode process 7111

 

 

1 ACCEPTED SOLUTION

avatar
New Contributor

I realize that netstat -lpn | grep 1022 command not display all port so I use netstat -nap | grep 1022 command I found a process run on this port


tcp 0 0 10.10.201.106:1022 10.10.201.10:2049 ESTABLISHED -

 

Not sure what process it is, seems OS process. So we try to restart the node I after it I find that port 1022 is free. Datanode is successfully started after it.

 

It is not good to set <1024 for any process, I'll plan to modify all non-standard port on our cluster to the standard one.

View solution in original post

1 REPLY 1

avatar
New Contributor

I realize that netstat -lpn | grep 1022 command not display all port so I use netstat -nap | grep 1022 command I found a process run on this port


tcp 0 0 10.10.201.106:1022 10.10.201.10:2049 ESTABLISHED -

 

Not sure what process it is, seems OS process. So we try to restart the node I after it I find that port 1022 is free. Datanode is successfully started after it.

 

It is not good to set <1024 for any process, I'll plan to modify all non-standard port on our cluster to the standard one.