Support Questions

Find answers, ask questions, and share your expertise

Dedicated Kafka Cluster: Does Sentry need HDFS?

avatar
Contributor

Hi, we have installed a "Kafka only" cluster on CDH 6.3 along with ZooKeeper, SMM, Sentry and SchemaRegistry.  During the cluster install, Sentry (from what I can remember) was dependent on HDFS being present.  Now that the cluster is online, does anyone know if it's safe to remove HDFS?

 

Thank you, 
Mike

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hi @mikejeezy ,

 

You shouldn't remove the HDFS component but you can stop the HDFS service in the scenario that you describe. 

 

Please refer to the documentation here: Configuring Kafka to Use Sentry Authorization

 

Sentry requires that your cluster include HDFS. After you install and start Sentry with the correct configuration, you can stop the HDFS service. 

 

Regards,

Steve

 

 

 

View solution in original post

4 REPLIES 4

avatar
Super Collaborator

Hi @mikejeezy ,

 

You shouldn't remove the HDFS component but you can stop the HDFS service in the scenario that you describe. 

 

Please refer to the documentation here: Configuring Kafka to Use Sentry Authorization

 

Sentry requires that your cluster include HDFS. After you install and start Sentry with the correct configuration, you can stop the HDFS service. 

 

Regards,

Steve

 

 

 

avatar
Explorer

Hey @StevenOD if we stop the service... do you know if there is a way to keep it from starting when we restart the cluster vi CM?

avatar
Super Collaborator

Hi @DataMike ,

 

I had a chat with some of my colleagues about this and it seems there is no easy way of stopping the HDFS starting when the cluster restarts. You might be able to do something via the Cloudera Manager API - but that probably quite complicated.

 

If it's any consolidation, this is fixed in the next generation of technology from Cloudera i.e. CDP. When you upgrade to CDP Sentry is replaced with Ranger and this HDFS dependency for Kafka no longer exists.

 

Regards,

Steve

avatar
Contributor

Thanks for checking @StevenOD  (sorry for posting that last post under a different account - forgot who I was logged in as).