question Re: Configure HDFS for SSL in Archives of Support Questions (Read Only)

Configure HDFS for SSL

melvinjoy_bandi — Wed, 03 Feb 2016 23:30:28 GMT

Hi,

I am trying to configure HDFS for SSL certificate and following the instructions on the document on End to End wire encryption with Knox.

Not sure how to set these properties in Ambari UI. Any help would be appreciated. Thanks

Use Ambari to set the following properties in core-site.xml.

[code language=”XML”] hadoop.ssl.require.client.cert=false hadoop.ssl.hostname.verifier=DEFAULT_AND_LOCALHOST hadoop.ssl.keystores.factory.class=org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory hadoop.ssl.server.conf=ssl-server.xml [/code]

Use Ambari to set the following properties in hdfs-site.xml.

[code language=”XML”] dfs.http.policy=HTTPS_ONLY dfs.datanode.https.address=workshop.hortonworks.com:50475 [/code]

The valid values for dfs.http.policy are HTTPS_ONLY & HTTP_AND_HTTPS.

The valid values for hadoop.ssl.hostname.verifier are DEFAULT, STRICT,STRICT_I6, DEFAULT_AND_LOCALHOST and ALLOW_ALL. Only use ALLOW_ALL in a controlled environment & with caution. And then use ambari to restart all hdfs services.

Re: Configure HDFS for SSL

aervits — Wed, 03 Feb 2016 23:30:49 GMT

in Ambari, go to HDFS service, then configs, in custom core-site.xml add the properties. In custom hdfs-site.xml add the hdfs-site specific properties @Melvin Bandi

Re: Configure HDFS for SSL

nsabharwal — Wed, 03 Feb 2016 23:33:12 GMT

@Melvin Bandi

Are you following this? https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_Security_Guide/content/ch_wire-https.html

FYI - knox gives SSL out of the box http://hortonworks.com/blog/end-end-wire-encryption-apache-knox/

Re: Configure HDFS for SSL

melvinjoy_bandi — Wed, 03 Feb 2016 23:40:36 GMT

Yes i am following this http://hortonworks.com/blog/end-end-wire-encryption-apache-knox/ to make a call to HDFS through KNOX security using user name and password. When i make a call through REST API it is showing this error

Unable to Connect to SSL Services due to PKIX Path Building Failed

Re: Configure HDFS for SSL

melvinjoy_bandi — Wed, 03 Feb 2016 23:54:17 GMT

In the below property can i give sandbox.hortonworks.com with the same port mentioned below or should i give Wedhdfs port?

dfs.datanode.https.address=workshop.hortonworks.com:50475

Re: Configure HDFS for SSL

melvinjoy_bandi — Wed, 03 Feb 2016 23:55:53 GMT

In the below property can i give sandbox.hortonworks.com with the same port mentioned below or should i give Wedhdfs port?

dfs.datanode.https.address=workshop.hortonworks.com:50475

Re: Configure HDFS for SSL

melvinjoy_bandi — Wed, 03 Feb 2016 23:56:32 GMT

*Webhdfs

Re: Configure HDFS for SSL

aervits — Wed, 03 Feb 2016 23:56:41 GMT

here are all suggested ports @Melvin Bandi link

Re: Configure HDFS for SSL

melvinjoy_bandi — Thu, 04 Feb 2016 00:14:34 GMT

I am getting this error when

SSL Between Knox & WebHDFS is configured as per the document End to End Wire Encryption with Apache Knox in Ambari

500 status codereceived on GET method for API: /api/v1/clusters/Sandbox/host_components?HostRoles/component_name=NAMENODE&metrics/dfs/FSNamesystem/HAState=active&fields=metrics/jvm/gcCount[1454512016,1454515616,15],metrics/jvm/GcCountConcurrentMarkSweep[1454512016,1454515616,15],metrics/jvm/GcTimeMillisConcurrentMarkSweep[1454512016,1454515616,15],metrics/rpc/NumOpenConnections[1454512016,1454515616,15],metrics/jvm/memHeapCommittedM[1454512016,1454515616,15],metrics/jvm/memHeapUsedM[1454512016,1454515616,15],metrics/cpu/cpu_system[1454512016,1454515616,15],metrics/cpu/cpu_user[1454512016,1454515616,15],metrics/cpu/cpu_nice[1454512016,1454515616,15],metrics/cpu/cpu_idle[1454512016,1454515616,15],metrics/cpu/cpu_wio[1454512016,1454515616,15],metrics/memory/mem_total[1454512016,1454515616,15],metrics/memory/mem_free[1454512016,1454515616,15],metrics/memory/mem_cached[1454512016,1454515616,15],metrics/rpc/RpcQueueTime_avg_time[1454512016,1454515616,15],metrics/rpc/RpcProcessingTime_avg_time[1454512016,1454515616,15]&format=null_padding

Error message: org.apache.ambari.server.controller.spi.SystemException: Error loading deferred resources

Re: Configure HDFS for SSL

melvinjoy_bandi — Thu, 04 Feb 2016 00:14:54 GMT

@Artem Ervits

Re: Configure HDFS for SSL

aervits — Thu, 04 Feb 2016 00:18:27 GMT

@Melvin Bandi check namenode service, connectivity on both client and server. Usually it means the server is not accepting connections from client.

Re: Configure HDFS for SSL

melvinjoy_bandi — Thu, 04 Feb 2016 00:24:38 GMT

Yes Namenode service seems to be stopped. I restarted it but the status seems to be Stopped.

NameNode	Stopped
SNameNode	Stopped
DataNodes	0/1 Started

Re: Configure HDFS for SSL

aervits — Thu, 04 Feb 2016 00:24:39 GMT

@Kevin Minder

Re: Configure HDFS for SSL

melvinjoy_bandi — Fri, 05 Feb 2016 02:57:05 GMT

@Kevin Minder Hi Kevin...Any update on this?

Re: Configure HDFS for SSL

aervits — Fri, 05 Feb 2016 04:41:20 GMT

@Melvin Bandi

I realize what the hold up is, this case is not tagged with security. I suggest you open a new case with this specific issue as the topic of this case is configuration. Tag the new case with security, SSL, etc.