Created 02-26-2021 09:30 AM
Hi community,
I trying to install Hadoop HA, but i have some problem to start datanode and nodemanager:
2021-02-26 17:53:48,971 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Shutdown complete.
2021-02-26 17:53:48,971 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.io.IOException: java.security.GeneralSecurityException: The property 'ssl.server.keystore.location' has not been set in the ssl configuration file.
at org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer.<init>(DatanodeHttpServer.java:199)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startInfoServer(DataNode.java:905)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1303)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:481)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2609)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2497)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2544)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2729)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2753)
Caused by: java.security.GeneralSecurityException: The property 'ssl.server.keystore.location' has not been set in the ssl configuration file.
at org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory.init(FileBasedKeyStoresFactory.java:152)
at org.apache.hadoop.security.ssl.SSLFactory.init(SSLFactory.java:148)
at org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer.<init>(DatanodeHttpServer.java:197)
... 8 more
2021-02-26 17:53:48,974 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2021-02-26 17:53:48,977 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG
But in my ssl-server.xml i correctly set this property:
<property>
<name>ssl.server.keystore.location</name>
<value>/etc/ssl/hadoop/keystore.jks</value>
</property>
<property>
<name>ssl.server.keystore.password</name>
<value>changeit</value>
<description>Must be specified.
</description>
</property>
<property>
<name>ssl.server.keystore.keypassword</name>
<value>changeit</value>
<description>Must be specified.
</description>
</property>
<property>
<name>ssl.server.keystore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
<property>
<name>ssl.server.exclude.cipher.list</name>
<value>TLS_ECDHE_RSA_WITH_RC4_128_SHA,SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA,
SSL_RSA_WITH_DES_CBC_SHA,SSL_DHE_RSA_WITH_DES_CBC_SHA,
SSL_RSA_EXPORT_WITH_RC4_40_MD5,SSL_RSA_EXPORT_WITH_DES40_CBC_SHA,
SSL_RSA_WITH_RC4_128_MD5</value>
<description>Optional. The weak security cipher suites that you want excluded
from SSL communication.</description>
</property>
In core-site.xml i set:
<property>
<name>hadoop.ssl.server.conf</name>
<value>/data/hadoop/etc/hadoop/ssl-server.xml</value>
</property>
<property>
<name>hadoop.ssl.client.conf</name>
<value>/data/hadoop/etc/hadoop/ssl-client.xml</value>
</property>
I see the services like namenode,resourcemanager started correcty but namenode and datanode doesn't started.
my hadoop version is: 2.8.5
my SO= centos 7
Do you have any suggestion to solve this problem?
Any help on this is really appreciated.
Created 03-02-2021 03:35 AM
Finally i found the problem:
2021-02-22 17:58:04,657 WARN org.apache.hadoop.hdfs.DFSUtil: SSL config ssl.server.truststore.location is missing. If dfs.https.server.keystore.resource is specified, make sure it is a relative path
I change my config to:
in hdfs-site.xml
<property>
<name>dfs.https.server.keystore.resource</name>
<value>ssl-server.xml</value>
</property>
<property>
<name>dfs.client.https.keystore.resource</name>
<value>ssl-client.xml</value>
</property>
in core-site.xml
<property>
<name>hadoop.ssl.server.conf</name>
<value>ssl-server.xml</value>
</property>
<property>
<name>hadoop.ssl.client.conf</name>
<value>ssl-client.xml</value>
</property>