Knox Gateway Start: Address already in use

[#1]

Hi there.

I'm setting up a small cluster of three nodes with ambari. The nodes are all fresh installed and only for HDP-usage.
During the setup, everything is working well. But when the Knox Gateway is started, the whole start-process is aborted due to a returning error:

2017-08-11 16:03:38,222 INFO  hadoop.gateway (GatewayServer.java:logSysProp(197)) - System Property: user.name=knox
2017-08-11 16:03:38,224 INFO  hadoop.gateway (GatewayServer.java:logSysProp(197)) - System Property: user.dir=/data/home/knox
2017-08-11 16:03:38,224 INFO  hadoop.gateway (GatewayServer.java:logSysProp(197)) - System Property: java.runtime.name=OpenJDK Runtime Environment
2017-08-11 16:03:38,224 INFO  hadoop.gateway (GatewayServer.java:logSysProp(197)) - System Property: java.runtime.version=1.8.0_141-b16
2017-08-11 16:03:38,224 INFO  hadoop.gateway (GatewayServer.java:logSysProp(197)) - System Property: java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre
2017-08-11 16:03:38,355 INFO  hadoop.gateway (GatewayConfigImpl.java:loadConfigResource(367)) - Loading configuration resource jar:file:/usr/hdp/2.6.1.0-129/knox/bin/../lib/gateway-server-0.12.0.2.6.1.0-129.jar!/conf/gateway-default.xml
2017-08-11 16:03:38,360 INFO  hadoop.gateway (GatewayConfigImpl.java:loadConfigFile(355)) - Loading configuration file /usr/hdp/2.6.1.0-129/knox/bin/../conf/gateway-site.xml
2017-08-11 16:03:38,380 INFO  hadoop.gateway (GatewayConfigImpl.java:initGatewayHomeDir(299)) - Using /usr/hdp/2.6.1.0-129/knox/bin/.. as GATEWAY_HOME via system property.
2017-08-11 16:03:38,695 INFO  hadoop.gateway (JettySSLService.java:init(95)) - Credential store for the gateway instance found - no need to create one.
2017-08-11 16:03:38,706 INFO  hadoop.gateway (JettySSLService.java:init(117)) - Keystore for the gateway instance found - no need to create one.
2017-08-11 16:03:38,709 INFO  hadoop.gateway (JettySSLService.java:logAndValidateCertificate(146)) - The Gateway SSL certificate is issued to hostname: xxx.yyy.zz.
2017-08-11 16:03:38,710 INFO  hadoop.gateway (JettySSLService.java:logAndValidateCertificate(149)) - The Gateway SSL certificate is valid between: 8/11/17 4:03 PM and 8/11/18 4:03 PM.
2017-08-11 16:03:38,945 INFO  hadoop.gateway (GatewayServer.java:startGateway(283)) - Starting gateway...
2017-08-11 16:03:39,054 FATAL hadoop.gateway (GatewayServer.java:main(155)) - Failed to start gateway: java.net.BindException: Address already in use (Bind failed) 

(Source: /var/log/knox/gateway.log)

So, "Address already in use" is the problem, which I try to solve. I assume that port 8443 is meant. I don't understand that. Nothing different than HDP is running on the machine, that could occupy the port, I also didn't change anything corresponding during the cluster-configuration, so all ports are the default ones and there shouldn't be any conflicts.

Can anybody help me with this problem?

[#2 · Accepted solution]

@Warius Unnlauf

Please check which other process is consuming that port?

# netstat -tnlpa | grep 8443

.

Sometimes we configure Ambari to run on HTTPS so the port might be same.

Using the netstat command we should see which Process ID is actually consuming that port.

[#3]

Of course! I also use HTTPS for Ambari-Web. With port 8443. I think that's the solution, great! Thank you.

Can I change the port by re-executing Step 3 in this article?

[#4]

@Warius Unnlauf

You do not need to re-execute Step3 to change the port. You can simply edit the ambari properties "client.api.ssl.port" to change the port from 8443 to something else.

# cp -f /etc/ambari-server/conf/ambari.properties /etc/ambari-server/conf/ambari.properties.BAK
# vi /etc/ambari-server/conf/ambari.properties

The Port property is the following thay needs to be changed ... followed by ambari-server restart.

client.api.ssl.port=8444

.

[#5]

Perfect. That did the trick. Works like a charm now.

Thank you!

[#6]

It's working fine now !! Thanks a lot
But I don't know how it was working before even a shutdown and restarted the cluster many times but didn't face this issue before, any suggestion ??

[#7]

@Warius Unnlauf If you want to change Knox port in order to resolve this, please change the property "gateway.port" in gateway-site.xml under Knox's conf directory. Sample config below:

<property>
      <name>gateway.port</name>
      <value>8883</value>
</property>

[#8]

I can see the property but not able to change it. Once you will change the port and try to start to Knox it will automatic change the port to the default port (8443) in the configuration file (gateway-site.xml).