Skip to main content
asmarz
asmarz
Silver ⭐️⭐️⭐️
January 27, 2020
Solved

Secure Webhdfs in Hadoop Hortonworks Cluster

Dear community

 

I have installed a hadoop cluster on 8 servers using Ambari Hortonworks.

I am able to access webhdfs using the ip address and the default port 50070 without authentication.

 

How can I secure Webhdfs?

 

P.S I did not enable using kerberos in Ambari > Enable kerberos , should I do it?

 

Any suggestion will be appreciated

Thanks

Asma

jsensharma
jsensharmaAnswer
Platinum ⭐️⭐️
February 3, 2020

@asmarz 

Good to know that your original issue is resolved.  However for any subsequent slightly different issue it is always better to open a new Community Thread that way the readers of this thread can easily find out One Error/Issue with one Solution.    Multiple issues in a single thread can cause readers to get confused.

.

If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

 

View in thread

10 replies

jsensharma
Platinum ⭐️⭐️
jsensharma
Platinum ⭐️⭐️
January 28, 2020

@asmarz 

Please refer to the following doc in order to know how you can enable SPNEGO authentication. Once you have enabled Kerberos for your cluster after that you can also enable the SPNEGO authentication. The following doc explains how to configure HTTP authentication for Hadoop components in a Kerberos environment.

 

By default, access to the HTTP-based services and UIs for the cluster are not configured to require authentication.  

1. https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/authentication-with-kerberos/content/authe_spnego_enabling_spnego_authentication_for_hadoop.html

2. https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/managing-and-monitoring-ambari/content/amb_start_kerberos_wizard_from_ambari_web.html

Translate
asmarz
asmarzAuthor
Silver ⭐️⭐️⭐️
January 30, 2020

thak you for your help

I tried to restart th ambari server but in vain .

I got this error


2020-01-30 18:20:21,866 INFO [main] KerberosChecker:64 - Checking Ambari Server Kerberos credentials.
2020-01-30 18:20:22,052 ERROR [main] KerberosChecker:120 - Client not found in Kerberos database (6)
2020-01-30 18:20:22,052 ERROR [main] AmbariServer:1119 - Failed to run the Ambari Server
org.apache.ambari.server.AmbariException: Ambari Server Kerberos credentials check failed.
Check KDC availability and JAAS configuration in /etc/ambari-server/conf/krb5JAASLogin.conf
at org.apache.ambari.server.controller.utilities.KerberosChecker.checkJaasConfiguration(KerberosChecker.java:121)
at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1110)

 

ht JAASLogin is configured like this

 

com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
renewTGT=false
doNotPrompt=true
useKeyTab=true
keyTab="/etc/security/ambariservername.keytab"
principal="ambariservername@REALM.COM"
storeKey=true
useTicketCache=false;
};

 

I tried to follow  these links 

https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/authentication-with-kerberos/content/kerberos_optional_install_a_new_mit_kdc.html

https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/authentication-with-kerberos/content/set_up_kerberos_for_ambari_server.html

 

Any suggestion please?

😞

Thanks

Translate
jsensharma
Platinum ⭐️⭐️
jsensharmaAnswer
Platinum ⭐️⭐️
February 3, 2020

@asmarz 

Good to know that your original issue is resolved.  However for any subsequent slightly different issue it is always better to open a new Community Thread that way the readers of this thread can easily find out One Error/Issue with one Solution.    Multiple issues in a single thread can cause readers to get confused.

.

If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

 

Translate
    willieken49
    willieken49
    Silver ⭐️
    February 4, 2020
    1. Set the value of the dfs.webhdfs.enabled property in hdfs-site.xml to true.

      <property>
      <name>dfs.webhdfs.enabled</name>
      <value>true</value>
      </property>

    2. Create an HTTP service user principal.

      kadmin: addprinc -randkey HTTP/$<Fully_Qualified_Domain_Name>@$<Realm_Name>.COM

      where:
      • Fully_Qualified_Domain_Name: Host where the NameNode is deployed.
      • Realm_Name: Name of your Kerberos realm.
    3. Create a keytab file for the HTTP principal.

      kadmin: xst -norandkey -k /etc/security/spnego.service.keytab HTTP/$<Fully_Qualified_Domain_Name>

    4. Verify that the keytab file and the principal are associated with the correct service.

      klist –k -t /etc/security/spnego.service.keytab

    5. Add the dfs.web.authentication.kerberos.principal and dfs.web.authentication.kerberos.keytab properties to hdfs-site.xml.

      <property>
      <name>dfs.web.authentication.kerberos.principal</name>
      <value>HTTP/$<Fully_Qualified_Domain_Name>@$<Realm_Name>.COM</value>
      </property>
      <property>
      <name>dfs.web.authentication.kerberos.keytab</name>
      <value>/etc/security/spnego.service.keytab</value>
      </property>

    6. Restart the NameNode and the DataNodes.
    Translate
    Terms & ConditionsAccessibility statement