Security issues in Hadoop HortonWorks after Enabling Kerberos

[#1]

Hello Community

My actual HDP version is :HDP-3.1.4.0

I have enabled Kerberos on the hadoop cluster but still the security officer checked a lot of vulnerabilitites

1.  we could access via these URLs the content without any authentication :

    http://ambarinode:61310/conf

    http://ambarinode:10002/conf

P.S : I did not install Hbase as a service

2. This URL also is accessible without authentication from all datanodes:

  http://datanode1:8042/conf

3. Grafana si also accessible without authentication 

   http://namenode:3000

   http://namenode:9060

Any ideas please

Thanks

Asma 

[#2]

@asmarz 

You might have enabled kerberos authentication for your cluster components. However, in order to secure the Web UIs offered by these components you will also need to enable the "SPNEGO Authentication".

By default, access to the HTTP-based services and UIs for the cluster are not configured to require authentication. Kerberos authentication can be configured for the Web UIs for HDFS, YARN, MapReduce2, HBase, Oozie, Falcon and Storm.   Please see [1] & [2]

1. Create a secret key used for signing authentication tokens.

dd if=/dev/urandom of=/etc/security/http_secret bs=1024 count=1
chown hdfs:hadoop /etc/security/http_secret
chmod 440 /etc/security/http_secret

2. Add additional properties for http authentication.

Example: in Advanced core-site:

hadoop.http.authentication.simple.anonymous.allowed =false
hadoop.http.authentication.signature.secret.file = /etc/security/http_secret
hadoop.http.authentication.type = kerberos
hadoop.http.authentication.kerberos.keytab = /etc/security/keytabs/spnego.service.keytab
hadoop.http.authentication.kerberos.principal = HTTP/_HOST@ EXAMPLE.COM
hadoop.http.filter.initializers = org.apache.hadoop.security.AuthenticationFilterInitializer
hadoop.http.authentication.cookie.domain = hortonworks.local

Once that is done then you will not be able to access those UIs without having a valid kerberos ticket. You will need to configure your web browser as mentioned in [3] in order to securely access those SPNEGO enabled component UIs.

- Similarly the following doc tells about how to enable HTTP Authentication for Ambari [4]

# ambari-server setup-kerberos
Using python  /usr/bin/python
Setting up Kerberos authentication
Enable Kerberos authentication [true|false] (false): true

[1] https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/authentication-with-kerberos/content/authe_spn...

[2] https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/authentication-with-kerberos/content/authe_spn...

[3] https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/authentication-with-kerberos/content/authe_spn...

[4] https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.0/bk_ambari-security/content/configuring_ambari_...

[#3]

Thank you for your pertinent answers as usual

so there is no another option except configuring browser with SPENGo for browsers?!

Is it recommander to configure the Hadoop cluster with https? If yes do you have a procedure please ? 

very appreciated 🙂

[#4]

@asmarz 

In order to enable SSL for various component you can refer to individual component docs.

following are some references:

1). Enabling HTTPS for Grafana & AMS
https://www.youtube.com/watch?v=dSH_9N94c4c
https://docs.cloudera.com/HDPDocuments/Ambari-2.7.3.0/using-ambari-core-services/content/amb_set_up_...
https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/using-ambari-core-services/content/amb_set_up_...

2). Enabling HTTPS for AmbariServer
https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/configuring-wire-encryption/content/set_up_ssl...

3). Enabling HTTPS for HDFS
https://community.cloudera.com/t5/Community-Articles/Enable-HTTPS-for-HDFS/ta-p/247181

4). Enabling HTTPS for various HDP services:
https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/configuring-wire-encryption/content/enabling_s...

.

.

If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
For different queries it is better to open new thread that way the responses are more organised.

[#5]

one last question please

should I use both kerberos and SSL for a secured cluster?

thanks