question Re: Atlas : how to secure Kafka ? in Archives of Support Questions (Read Only)

Atlas : how to secure Kafka ?

smartdatabundle — Tue, 01 Aug 2017 14:08:24 GMT

Hello,

I could access all topics on kafka without authentification.

My question : how I could secure access on kafka topics ?

Thks.

Re: Atlas : how to secure Kafka ?

gmartin — Tue, 01 Aug 2017 14:57:15 GMT

@Smart Data

Atlas is more Governance related, security to a less extent.

You secure Kafka via Kerberos for authentication, and Ranger for authorization:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/index.html#bk_security

Re: Atlas : how to secure Kafka ?

smartdatabundle — Tue, 01 Aug 2017 15:09:26 GMT

@Graham Martin

Thks for your quick reply. Is there an alternative to Kerberos ? May be Apache Knox + LDAP ?

I went to the link: indeed, it explains only the use of Kerberos.

Re: Atlas : how to secure Kafka ?

gmartin — Tue, 01 Aug 2017 15:28:38 GMT

@Smart Data

Ranger can be used to sync users with LDAP/AD. Credentials are stored in LDAP/AD, and Ranger configured to access.

Knox is used as a proxy, but more for REST API service calls, and some UIs. It is not meant to proxy high volume traffic like Kafka messages.

Re: Atlas : how to secure Kafka ?

smartdatabundle — Tue, 01 Aug 2017 16:42:08 GMT

@Graham Martin

Thks for your expanation. I am going to install & to use Kerberos.

Re: Atlas : how to secure Kafka ?

Shelton — Tue, 01 Aug 2017 17:24:32 GMT

@@Smart Data

If you intend to run a secure Hadop cluster then there is no way you can avoid Kerberos. Below are the difference between knox and kerberos.

The Apache Knox Gateway is a system that provides a single point of authentication and access. It provides the following features:

Single REST API Access Point
Centralized authentication, authorization and auditing for Hadoop REST/HTTP services
LDAP/AD Authentication, Service Authorization and Audit
Eliminates SSH edge node risks
Hides Network Topology

LAYERS OF DEFENSE FOR A HADOOP CLUSTER

Perimeter Level Security – Network Security, Apache Knox (gateway)
Authentication : Kerberos
Authorization
OS Security : encryption of data in network and HDFS

Apache Knox can also access a Hadoop cluster over HTTP or HTTPS

CURRENT FEATURES OF APACHE KNOX

Authenticate : by LDAP or Cloud SSO Provider
Provides services for HDFS, HCat, HBase, Oozie, Hive, YARN, and Storm
HTTP access for Hive over JDBC support is available (ODBC driver Support- In Future)

Hope that helps to explain.

Re: Atlas : how to secure Kafka ?

Shelton — Tue, 01 Aug 2017 17:25:01 GMT

@@Smart Data

If you intend to run a secure Hadop cluster then there is no way you can avoid Kerberos. Below are the difference between knox and kerberos.

The Apache Knox Gateway is a system that provides a single point of authentication and access. It provides the following features:

Single REST API Access Point
Centralized authentication, authorization and auditing for Hadoop REST/HTTP services
LDAP/AD Authentication, Service Authorization and Audit
Eliminates SSH edge node risks
Hides Network Topology

LAYERS OF DEFENSE FOR A HADOOP CLUSTER

Perimeter Level Security – Network Security, Apache Knox (gateway)
Authentication : Kerberos
Authorization
OS Security : encryption of data in network and HDFS

Apache Knox can also access a Hadoop cluster over HTTP or HTTPS

CURRENT FEATURES OF APACHE KNOX

Authenticate : by LDAP or Cloud SSO Provider
Provides services for HDFS, HCat, HBase, Oozie, Hive, YARN, and Storm
HTTP access for Hive over JDBC support is available (ODBC driver Support- In Future)

Hope that helps to explain.

Re: Atlas : how to secure Kafka ?

smartdatabundle — Tue, 01 Aug 2017 19:22:18 GMT

@Geoffrey Shelton Okot thks for this explanation.