Skip to main content
mikejeezy
mikejeezy
Gold ⭐️
April 9, 2020
Solved

Dedicated Kafka Cluster: Does Sentry need HDFS?

Hi, we have installed a "Kafka only" cluster on CDH 6.3 along with ZooKeeper, SMM, Sentry and SchemaRegistry.  During the cluster install, Sentry (from what I can remember) was dependent on HDFS being present.  Now that the cluster is online, does anyone know if it's safe to remove HDFS?

 

Thank you, 
Mike

StevenOD
StevenODAnswer
Platinum ⭐️
April 10, 2020

Hi @mikejeezy ,

 

You shouldn't remove the HDFS component but you can stop the HDFS service in the scenario that you describe. 

 

Please refer to the documentation here: Configuring Kafka to Use Sentry Authorization

 

Sentry requires that your cluster include HDFS. After you install and start Sentry with the correct configuration, you can stop the HDFS service. 

 

Regards,

Steve

 

 

 

View in thread

4 replies

StevenOD
Platinum ⭐️
StevenODAnswer
Platinum ⭐️
April 10, 2020

Hi @mikejeezy ,

 

You shouldn't remove the HDFS component but you can stop the HDFS service in the scenario that you describe. 

 

Please refer to the documentation here: Configuring Kafka to Use Sentry Authorization

 

Sentry requires that your cluster include HDFS. After you install and start Sentry with the correct configuration, you can stop the HDFS service. 

 

Regards,

Steve

 

 

 

Translate
DataMike
DataMike
Silver ⭐️⭐️
April 13, 2020

Hey @StevenOD if we stop the service... do you know if there is a way to keep it from starting when we restart the cluster vi CM?

Translate
StevenOD
Platinum ⭐️
StevenOD
Platinum ⭐️
April 16, 2020

Hi @DataMike ,

 

I had a chat with some of my colleagues about this and it seems there is no easy way of stopping the HDFS starting when the cluster restarts. You might be able to do something via the Cloudera Manager API - but that probably quite complicated.

 

If it's any consolidation, this is fixed in the next generation of technology from Cloudera i.e. CDP. When you upgrade to CDP Sentry is replaced with Ranger and this HDFS dependency for Kafka no longer exists.

 

Regards,

Steve

Translate
Terms & ConditionsAccessibility statement