How can I resolve java.net.BindException: Address already in use Error?

[#1]

2014-08-27 10:50:47,196 WARN component.AbstractLifeCycle (AbstractLifeCycle.java:setFailed(204)) - FAILED SelectChannelConnector@0.0.0.0:4040: java.net.BindException: Address already in use

java.net.BindException: Address already in use

[#2]

What does netstat -nap | grep 4040 gives. you will get the output with the process id . Check what process it is.

[#3 · Accepted solution]

When a spark context is created, it starts an application UI on port 4040 by default. When the UI starts, it checks to see if the port is in use, if so it should increment to 4041. Looks like you have something running on port 4040 there. The application should show you the warning, then try to start the UI on 4041.

This should not stop your application from running. If you really want to get around the WARNING, you can manually specify which port for the UI to start on, but I would strongly advise against doing so.

To manually specify the port, add this to your spark-submit:

--conf spark.ui.port=your_port

[#4]

Hi,

 can I instead add the following line to spark-defaults.conf file:

spark.ui.port   4041

Will that have the same effect ?

Thanks

[#5]

At time, due to incomplete shutdown, previous process still may be using the port. So just check the process that is using the port and (if it is the same process you are trying to start, which it is in 99.9% of the cases) then just kill the previous process before starting.

netstat -lnap | grep <port>
#The output has the process id in it
kill -9 <pid> 

# After the process is killed, try starting the service.