hi all we have production cluster with HDP - 2.6.4 version we have 186 data-node machines ( DELL MACHINES WITH 10 disks ) we try to re balance the data on the disks so disks will be with the same used size but without success we feel that 2.6.4 version not have the tools that support re balance!!! as I mentioned on each data-node machine we have 10 disks while each disk is 1.8T and some of the disks are 55% used and some of them are only 1% used so we have non balanced disks ( its like some disk are not useful ) , but why HDFS not balanced the data on all disks?? my question - from which HDP version , we can re balance the data-node disks ? dose 2.6.5 version support re balance ? or from 3.X ? please advice , what we can do ? as I mentioned this is very huge cluster and we get the bad feeling that the current HDP version ( 2.6.4 ) not support any re balance - is it true? example /dev/sdc                 3842878616 357409860 3485452372  10% /data_hdfs/sdc /dev/sde                 3842878616 460433776 3382428456  42% /data_hdfs/sde /dev/sdi                 3842878616   8606628 34255604   1% /data_hdfs/sdi /dev/sdg                 3842878616 256937520 85924712   7% /data_hdfs/sdg /dev/sdd                 3842878616 465520852 3377341380  53% /data_hdfs/sdd /dev/sdh                 3842878616     90136 42772096   1% /data_hdfs/sdh /dev/sdb                 3842878616 466423860 3376438372  53% /data_hdfs/sdb ... View more

hi all we have small cluster ( ambari version - 2.6.1 ) 3 masters machines 3 kafka machines 3 workers machines is it possible to removed completely the HDP , for new fresh HDP version ? Now we have HDP - 2.6.5 And we want to install new HDP - 2.6.4 ... View more

hi all is it necessary to restart the ambari server after we delete ambari service by REST API ? I as asking because after we delete some service from the ambari by API we saw that service really Disappeared from ambari dartboard so I not sure if we need also to restart the ambari server to take affect? ... View more

hi all under /var/log/zookeeper , we have the zookeeper logs , as the following -rw-r--r-- 1 zookeeper hadoop 104857712 Jul  5 15:54 zookeeper-zookeeper-server-master.sys87.com.log.1 -rw-r--r-- 1 zookeeper hadoop       177 Jul 16 11:57 zookeeper-zookeeper-server-master.sys87.com.out -rw-r--r-- 1 zookeeper hadoop  66358998 Jul 16 23:15 zookeeper-zookeeper-server-master.sys87.com.log the problem is that we get only the INFO severity from the logs! and when we set the TRCE severity , and restart the zookeeper service , then its not change the zookeeper log content to trace mode, can we get example of log4j for zookeeper ? we suspect that our log4j isn't configured properly ... View more

hi all we have two spark thrift servers component in our ambari cluster one thrift server is on master01 machine and the second is on master03 machine how to stop the thrift server component on master01 and master03 machine by REST API ... View more

hi all by the following API we can delete the thrift server component on master01 and master03 nodes # curl -iv -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://$AMBARI_SERVER:8080/api/v1/clusters/hdp/hosts/master01/host_components/SPARK2_THRIFTSERVER # curl -iv -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://$AMBARI_SERVER:8080/api/v1/clusters/hdp/hosts/master03/host_components/SPARK2_THRIFTSERVER but how to know by REST API that thrift server component , not defined on specific node ( after deletion ) ... View more