Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hadoop cluster setup in azure with HDI

Hadoop cluster setup in azure with HDI

I want to install HDP in the azure with the use of HDI , I am new for this activity . I have below query installing the Hadoop Cluster in the HDI Azure :

  • Why there are multiple cluster types ?
  • if I setup Hadoop cluster there are spark , hbase , ranger .. etc these services are missing , If I want to install spark within the Hadoop cluster or other services how to do that ?
  • how to implement the security like in hdp we have kerberos , ranger ..etc. ?

Thanks

1 REPLY 1
Highlighted

Re: Hadoop cluster setup in azure with HDI

Hi @Anurag Mishra,

HDI is used for ephemeral clusters based on a finite set of services. It's primary purpose is to quickly set up the services, run a workload, and then bring the cluster down. HDI was not designed to handle long-running workloads, or production data lake architecture. Finally, HDI is not configurable. You only have features provided in the images.

With HDI security is an additional cost. You will need to leverage both Ranger as well as Azure Active Directory.

If you would like more control and more of a production-ready environment, I'd suggest running HDP as IaaS (Infrastructure as a Service). This can be quickly and easily provisioned using Cloudbreak https://hortonworks.com/open-source/cloudbreak/

Hope this helps.

Don't have an account?
Coming from Hortonworks? Activate your account here