Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Dedicated Kafka Cluster: Does Sentry need HDFS?

Solved Go to solution

Dedicated Kafka Cluster: Does Sentry need HDFS?

Contributor

Hi, we have installed a "Kafka only" cluster on CDH 6.3 along with ZooKeeper, SMM, Sentry and SchemaRegistry.  During the cluster install, Sentry (from what I can remember) was dependent on HDFS being present.  Now that the cluster is online, does anyone know if it's safe to remove HDFS?

 

Thank you, 
Mike

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Dedicated Kafka Cluster: Does Sentry need HDFS?

Expert Contributor

Hi @mikejeezy ,

 

You shouldn't remove the HDFS component but you can stop the HDFS service in the scenario that you describe. 

 

Please refer to the documentation here: Configuring Kafka to Use Sentry Authorization

 

Sentry requires that your cluster include HDFS. After you install and start Sentry with the correct configuration, you can stop the HDFS service. 

 

Regards,

Steve

 

 

 

View solution in original post

4 REPLIES 4
Highlighted

Re: Dedicated Kafka Cluster: Does Sentry need HDFS?

Expert Contributor

Hi @mikejeezy ,

 

You shouldn't remove the HDFS component but you can stop the HDFS service in the scenario that you describe. 

 

Please refer to the documentation here: Configuring Kafka to Use Sentry Authorization

 

Sentry requires that your cluster include HDFS. After you install and start Sentry with the correct configuration, you can stop the HDFS service. 

 

Regards,

Steve

 

 

 

View solution in original post

Highlighted

Re: Dedicated Kafka Cluster: Does Sentry need HDFS?

Explorer

Hey @StevenOD if we stop the service... do you know if there is a way to keep it from starting when we restart the cluster vi CM?

Highlighted

Re: Dedicated Kafka Cluster: Does Sentry need HDFS?

Expert Contributor

Hi @DataMike ,

 

I had a chat with some of my colleagues about this and it seems there is no easy way of stopping the HDFS starting when the cluster restarts. You might be able to do something via the Cloudera Manager API - but that probably quite complicated.

 

If it's any consolidation, this is fixed in the next generation of technology from Cloudera i.e. CDP. When you upgrade to CDP Sentry is replaced with Ranger and this HDFS dependency for Kafka no longer exists.

 

Regards,

Steve

Highlighted

Re: Dedicated Kafka Cluster: Does Sentry need HDFS?

Contributor

Thanks for checking @StevenOD  (sorry for posting that last post under a different account - forgot who I was logged in as).

Don't have an account?
Coming from Hortonworks? Activate your account here