This article describe important configuration steps to consider for securing the Hortonworks Data Platform (HDP) Hadoop clusters that are created from Azure marketplace images of HDP. The current HDP offering on Azure utilizes a template that locks down all ports and protocols by default, except for port 22, for security reasons. This article explains the methods to secure and limit access to the service endpoints (for example those of Ambari) to known IP addresses from the Internet from this default configuration.
Here are the steps to create an Azure IaaS cluster:
- Login to your Azure portal and bring up the Marketplace listings:
2. Find the Hortonworks listing.
3. Fill in the information to create a cluster.
4. Fill in the details for the security information. We recommend using SSH keys instead of passwords for direct access to the nodes, and a strong password for Ambari.
5. Accept the terms and launch the cluster.
6. Wait for the cluster deployment to complete.
7. If you need to access the cluster via ssh, it should already be setup using the credentials you configured in the above steps.
8. If you need access to Ambari, you need to explicitly open up the Ambari ports. You have two choices:
- Create an SSH tunnel using the instructions in Microsoft Azure blog at https://blogs.msdn.microsoft.com/pliu/2017/01/17/ssh-tunnel-to-endpoints-in-azure-vnet-from-windows/
- Open up port 8080 for access by external clients. For doing this:
A. locate the Network security group associated with the cluster, typically named as <clustername>-nsg.
B. view the settings for the NSG
C. Add an inbound rule, paying special attention to the CIDR block and the port number. The CIDR block below should use your client IP address range. If you want to enable world access, you can use 0.0.0.0/0 for the CIDR block, but please note that this leaves the port wide open for attacks by any malicious agent on the internet.
D. once you have enabled the inbound rule you should be able to bring up Ambari at port 8080 on your cluster
Additionally, following security measures should also be considered:
- Secure your cluster via kerberization, which will ensure strong user authentication is performed before granting any access to cluster resources/services/data and implement industry standard security configurations as recommended in HDP security guide: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_Security_Guide/content/ch_hdp-security-g...
- To avoid on-the-wire password sniffing, ensure Ambari is configured to run with https protocol with secure ciphers. For more details, please refer to the above security guide.
