I've initially installed 2 slave nodes(node01, node02) cluster using Ambari Cluster Install wizard successfully.
My Ambari Server is running on other node ( node 00 ).
Problem is at the time of install wizard i gave only 2 slave node (because it asked me to enter the target hosts)
so i assumed and gave node 01, node 02(slaves (say)),
Eventually by default node 01 and node 02 behaved as master and slave (All the services/components were shared by these 2 nodes (node 01 and 02) and it went successfully and became 2 node cluster
Now what is my requirement is want to add node 00 (Ambari server is running) to make whole system as 3 node cluster and want to make node 00 as master (if there are any possibilities) and or else simply add and run few of services on it.
One thing all the services were shared between node 01 and node 02, so what services need to add for node 00.
shall i make node 00 as data node/name node/sec. name node? if yes how?
please help me.
1. If this is test cluster and if you are using for your own purpose then I will suggest to add node00 to the cluster, delete all service using api and install/configure the services again.
2. Qu - "One thing all the services were shared between node 01 and node 02, so what services need to add for node 00."
-- It depends upon your compute on all nodes [ie. how much memory cpu and HDD you are having]. As mentioned above if this is a test cluster then below is recommended config -
A. node00 - All Master services[Ambari,NN,YARN,ZK,DN,NM]
node01 - Master/Slave [SNN,DN,NM,ZK[optional]
node02 - Slave DN,NM,ZK[optional]
3. If you want to stay with current setup then install services on node00 as mentioned below -
- node00 - DN,NM,ZK[optional]
So let me know if you have any more questions. Will be happy to help.
Once you add the node to the cluster you can install the services on the node by -
The Ambari interface provides a wizard that makes it easy to add nodes. The "Add hosts" wizard in Ambari assumes that these nodes will be workers (slaves) - so it gives the options to install DataNode, NodeManager and client services. Start the wizard from the Hosts tab, then click "actions" and "Add New Hosts" - as shown.
After adding nodes, you can then move master services from one node to another. For example, go to Services >> HDFS, and from the service actions button top right (see screenshot) you will see an option to move the NameNode service.
You can move services around as you wish - just be aware that doing so will force a small amount of downtime on your cluster. Ambari will tell you which services need restarting afterwards. Hope that helps.
In a Hadoop cluster, Spark processing gets run on top of YARN as a set of distributed processes. So at the very simplest level, Spark will run wherever NodeManager (the slave component of YARN) is installed.
Hive queries get compiled into MapReduce (or Tez or possibly Spark) jobs by the HiveServer. These jobs also run on top of YARN. So again at a very simple level, Hive queries will run wherever NodeManager is installed.
Hive also uses master services. You can move the Hive master services from one master node to another if you need to. These master services are HiveServer2 (which looks after client connections and compilation), Hive Metastore (which stores the metadata) and WebHCat Server (=HCatalog).
This is a very high-level view. Does any of this help?