Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDF Version: 2.1.2

Test Environment:

Single AWS VPC with multiple subnets, EC2 Security Groups with port rules controlling access between subnets, AWS LogFlow to track accepted/rejected traffic, 1 EC2 instance for Management, 3 EC2 instances for worker nodes, Laptop on public IP for remote access.

Deployment / Testing Method:

  1. Installed baseline of Ambari, Ambari Infra, Ambari Metrics, NiFi Certificate Authority, Zookeeper Cluster, NiFi Cluster, Clients
  2. Added Ranger
  3. Enabled SSL across all services
  4. Tested all commonly used interfaces, checked for rejected traffic

Not Tested:

  • Using External LDAP/AD services
  • Using external Certificate Authority
  • Connecting to Auxiliary services

Deployment Topology

ZoneM(anagement)D(ata)R(emote)P(ublic)
MembersManagement NodeWorker Nodes 1,2,3Other NiFi ClusterUsers
ServicesAmbari Infra (Infra Solr), Ranger, Metrics Collector, Grafana, Metrics Monitor, Nifi Certificate Authority, Clients (Infra Solr, ZooKeeper)NiFi, Zookeeper, Metrics Monitor, Clients (Zookeper)NiFiBrowser, ssh client

Firewall Rules:

Source ZoneDest ZonePortNotes
MD22ssh, if used for deployment
MD8670Ambari Agent
MD3000, 61300, 61310, 61330, 61320, 61388, 61288, 61181, 2181, 60200, 6188Ambari Metrics Service
MD2181, 61181ZooKeeper
DM8080 (http), 8443 (https)Ambari Interface
DM8440, 8441Ambari Agents
DM6182, 6080, 8886Ranger Services
DM3000, 61300, 61310, 61330, 61320, 61388, 61288, 61181, 2181, 60200, 6188Ambari Metrics Service
PM8080 (http), 8443 (https)Ambari
PM6080Ranger
PM3000Grafana
PM8886Solr Admin
PM22ssh
PD22ssh (optional)
PD9090 (http), 9091(https)NiFi Interface
DR9090 (http), 9091(https)NiFi Interface & Data Transfer

Additional Port Considerations:

  • Ports for all Zones to connect to LDAP/AD if used
  • Ports for all Zones to send Logging and Alerts (smtp etc.) to other systems
  • Ports for NiFi to connect to target systems, e.g. HDFS, Hive, Kafka, etc.
  • You will require access to your CA to generate and move certificates; it is probably not necessary to open a Port for direct connection
1,435 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎03-23-2017 04:05 PM
Updated by:
 
Contributors
Top Kudoed Authors