Hi Madhura,   The following link may be usefull for your question https://docs.cloudera.com/HDPDocuments/HDP2/HDP-2.0.6.0/bk_installing_manually_book/content/rpm-chap1-11.html   For calculating  maximum number of containers allowed per node. The following formula can be used: # of containers = min (2*CORES, 1.8*DISKS, (Total available RAM) / MIN_CONTAINER_SIZE) Where MIN_CONTAINER_SIZE is the minimum container size (in RAM). This value is dependent on the amount of RAM available -- in smaller memory nodes, the minimum container size should also be smaller   Thanks AKR ... View more

Hi @ppatel, As this thread is older and was marked 'Solved' in March of 2019 you would have a better chance of receiving a resolution by starting a new thread. This will also provide the opportunity to provide details about your script that could aid others in providing a more relevant answer to your question.      ... View more

@Madhura Mhatre The hdfs balancer utility that analyzes block placement and balances data across the DataNodes. The balancer moves blocks until the cluster is deemed to be balanced. A threshold parameter is a float number between 0 and 100 (12.5 for instance). From the average cluster utilization (about 50% in the graph below), the balancer process will try to converge all data nodes' usage in the range [average - threshold, average + threshold]. In the current example: - Higher (average + threshold): 60% if run with the default threshold (10%) - Lower (average - threshold): 40% You can easily notice that the smaller your threshold, the more balanced your data nodes will be. For very small threshold, the cluster may not be able to reach the balanced state if other clients concurrently write and delete data in the cluster. A threshold of 15 should be okay $ hdfs balancer -threshold 15 or by giving the list of datanodes $ hdfs balancer -threshold 15 -include hostname1,hostname2,hostname3 HTH ... View more

@Madhura Mhatre If you increase the size of the same ibdata disk mount then you don't need to update any metadata because the pointers will be intact in the metastore. Make sure you shut down the all the databases on the mount point before increasing the size. Happy hadooping ... View more

@Geoffrey Shelton Okot Thank you. I think Decommissioning is a best way to do this. Do I need to turn on maintainance mode while I do decommissioning datanode? Also Do I need to stop the services like - NodeManager, Ambari metrics for that datanode after decommissioning? ... View more

@Madhura Mhatre Can you share the output of the rebalancer? Have you tried reducing gradually from 25% - 20% -15% -10% to see if you are going some space Depending on the purpose of the cluster, you shouldn't attempt this on production. You can remove the directory through the Ambari property: dfs.datanode.data.dir and do a rolling restart of data nodes. Make sure you don't have any missing/under-replicated blocks before restarting the new data node. umount the /data1 format it remount ... View more

I see Spark2 history server available in my cluster. Can you check if Spark2 history server is already installed on that node. Please refer the screenshot For clients, move option is not required. You can install clients on the new node in the similar way. You will see "Spark client" in the dropdown when you click +ADD ... View more

@Madhura Mhatre, 1) If you have multiple hiveservers, you will have multiple znodes(of the above mentioned forma) under /hiveserver2. When you stop the hiveserver2, respective znode will be deleted from zookeeper. If you stop all the hiveserver instances, then /hiveserver2 will have no znodes under it. You should not delete /hiveserver2 . hiveserver2 is the parent znode , respective hiveserver znodes will be under the parent znode ie /hiveserver2. 2) By installing do you mean you want to add another hiveserver2 to the cluster. If yes, then follow the below doc link https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_hadoop-high-availability/content/additional_hs2_installed_with_ambari.html. ... View more

Hi @Jay Kumar SenSharma @Madhura Mhatre else You can delete requested services directly from CLI by using API call. It will list components of services curl -u admin -H "X-Requested-By:ambari" -X GET http://<ambari-server_host>/api/v1/clusters/<clustername>/hosts/<fqdn_of_requested_node_to_delete_service>; EX: curl -u admin -H "X-Requested-By:ambari" -X GET http://<ambari-server_host>/api/v1/clusters/<clustername>/hosts/fdlslr7v211.pwcccorp.net To Delete service: curl -u admin -H "X-Requested-By:ambari" -X DELETE http://<ambari-server_host>/api/v1/clusters/<clustername>/hosts/<fqdn_of_requested_node_to_delete_service>/<SERVICE_NAME>; Example to delete HST_AGENT: curl -u admin -H "X-Requested-By:ambari" -X GET http://<ambari-server_host>/api/v1/clusters/<clustername>/hosts/fdlslr7v211.pwcccorp.net/HST_AGENT ... View more