Configure HDFS for SSL

[#1]

Hi,

I am trying to configure HDFS for SSL certificate and following the instructions on the document on End to End wire encryption with Knox.

Not sure how to set these properties in Ambari UI. Any help would be appreciated. Thanks

Use Ambari to set the following properties in core-site.xml.

[code language=”XML”] hadoop.ssl.require.client.cert=false hadoop.ssl.hostname.verifier=DEFAULT_AND_LOCALHOST hadoop.ssl.keystores.factory.class=org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory hadoop.ssl.server.conf=ssl-server.xml [/code]

Use Ambari to set the following properties in hdfs-site.xml.

[code language=”XML”] dfs.http.policy=HTTPS_ONLY dfs.datanode.https.address=workshop.hortonworks.com:50475 [/code]

The valid values for dfs.http.policy are HTTPS_ONLY & HTTP_AND_HTTPS.

The valid values for hadoop.ssl.hostname.verifier are DEFAULT, STRICT,STRICT_I6, DEFAULT_AND_LOCALHOST and ALLOW_ALL. Only use ALLOW_ALL in a controlled environment & with caution. And then use ambari to restart all hdfs services.

[#2 · Accepted solution]

in Ambari, go to HDFS service, then configs, in custom core-site.xml add the properties. In custom hdfs-site.xml add the hdfs-site specific properties @Melvin Bandi

[#3]

In the below property can i give sandbox.hortonworks.com with the same port mentioned below or should i give Wedhdfs port?

dfs.datanode.https.address=workshop.hortonworks.com:50475

[#4]

*Webhdfs

[#5]

@Melvin Bandi

Are you following this? https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_Security_Guide/content/ch_wire-https.htm...

FYI - knox gives SSL out of the box http://hortonworks.com/blog/end-end-wire-encryption-apache-knox/

[#6]

Yes i am following this http://hortonworks.com/blog/end-end-wire-encryption-apache-knox/ to make a call to HDFS through KNOX security using user name and password. When i make a call through REST API it is showing this error

Unable to Connect to SSL Services due to PKIX Path Building Failed

[#7]

In the below property can i give sandbox.hortonworks.com with the same port mentioned below or should i give Wedhdfs port?

dfs.datanode.https.address=workshop.hortonworks.com:50475

[#8]

here are all suggested ports @Melvin Bandi link

[#9]

I am getting this error when

SSL Between Knox & WebHDFS is configured as per the document End to End Wire Encryption with Apache Knox in Ambari

500 status codereceived on GET method for API: /api/v1/clusters/Sandbox/host_components?HostRoles/component_name=NAMENODE&metrics/dfs/FSNamesystem/HAState=active&fields=metrics/jvm/gcCount[1454512016,1454515616,15],metrics/jvm/GcCountConcurrentMarkSweep[1454512016,1454515616,15],metrics/jvm/GcTimeMillisConcurrentMarkSweep[1454512016,1454515616,15],metrics/rpc/NumOpenConnections[1454512016,1454515616,15],metrics/jvm/memHeapCommittedM[1454512016,1454515616,15],metrics/jvm/memHeapUsedM[1454512016,1454515616,15],metrics/cpu/cpu_system[1454512016,1454515616,15],metrics/cpu/cpu_user[1454512016,1454515616,15],metrics/cpu/cpu_nice[1454512016,1454515616,15],metrics/cpu/cpu_idle[1454512016,1454515616,15],metrics/cpu/cpu_wio[1454512016,1454515616,15],metrics/memory/mem_total[1454512016,1454515616,15],metrics/memory/mem_free[1454512016,1454515616,15],metrics/memory/mem_cached[1454512016,1454515616,15],metrics/rpc/RpcQueueTime_avg_time[1454512016,1454515616,15],metrics/rpc/RpcProcessingTime_avg_time[1454512016,1454515616,15]&format=null_padding

Error message: org.apache.ambari.server.controller.spi.SystemException: Error loading deferred resources

[#10]

@Artem Ervits