Edge node setup

@irfangk1 

Edge nodes are the interface between the Hadoop cluster and the outside network. They’re also often used as staging areas for data being transferred into the Hadoop cluster. Installing the edge node is as easy as adding a node to the cluster. The only difference is that on the edge-node you will only deploy client software ONLY e.g SQOOP, PIG, HDFS, YARN, HBase, SPARK, ZK HIVE or HUE etc to enable you to for example to run HDFS commands on the edge-node.

To enable communication between the outside network and the Hadoop cluster, edge nodes need to be multi-homed into the private subnet of the Hadoop cluster as well as into the corporate network.

A multi-homed computer is one that has dedicated connections to multiple networks. This is a practical illustration of why edge nodes are perfectly suited for interaction with the world outside the Hadoop cluster. Keeping your Hadoop cluster in its own private subnet is an excellent practice, so these edge nodes serve as a controlled window inside the cluster

If you're using Knox for perimeter security, then all clients' software should reside on a dedicated Knox gateway machine to which end users can submit their requests.It's good practice to divide the cluster into master nodes, worker nodes, edge node(s), and management node.
Services such as Namenode, Zookeeper, Yarn Resource Manager, Secondary Namenode usually run on the master node machines. Worker nodes aka Datanode should be further divided into two categories those running HDFS and Yarn and those running Storm and Kafka and other components

A minimum best practice is to have 3-5 master and >5 data nodes.

HTH