we have ambari cluster we haven't started using HDFS but we get service-level alert is triggered if the increase in storage capacity usage - why? please advice why? why capacity is grow in spite we not use HDFS? and what is the solotion for this? ... View more

hi all python yarn-utils.py script is a nice tool that Determining HDP Memory Configuration Settings background - https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_command-line-installation/content/determine-hdp-memory-config.html I want to know about other tools that can help us to perform fine tuning to the ambari cluster there are many other parameters that python yarn-utils.py isn't covered and need to set them according to the HW system for example - yarn.nodemanager.resource.cpu-vcores and many many others please help us to know if there are other tools that can help us to give the right values for many aparmters and by this way we can set them in the ambari cluster ... View more

as I read from the doc ,yarn.nodemanager.resource.cpu-vcores are by default ~80% of total vCPUs available on the machine so just to clear that in case we have ambari cluster with 3 master machine and 3 workers )( machines , and each worker have 8 core then the calculate will be 3X8=32 ? or maybe yarn.nodemanager.resource.cpu-vcores should be calculate per worker machine as: 1X8=8 ? or else? ... View more

we have ambari cluster with 3 masters machine and 4 datanode machine we run the hdfs dfsadmin -report and we found Missing blocks: 4 how to know the reason for these missing blocks? second what is the workaround that we need to do regarding that? hdfs dfsadmin -report Configured Capacity: 8226130288640 (7.48 TB) Present Capacity: 8225526102776 (7.48 TB) DFS Remaining: 8205621209848 (7.46 TB) DFS Used: 19904892928 (18.54 GB) DFS Used%: 0.24% Under replicated blocks: 4 Blocks with corrupt replicas: 0 Missing blocks: 4 Missing blocks (with replication factor 1): 0 ------------------------------------------------- Live datanodes (4): Name: 10.164.252.32:50010 (worker03.sys76.com) Hostname: worker03.sys76.com Decommission Status : Normal Configured Capacity: 1170504683520 (1.06 TB) DFS Used: 5715611648 (5.32 GB) Non DFS Used: 0 (0 B) DFS Remaining: 1164727208338 (1.06 TB) DFS Used%: 0.49% DFS Remaining%: 99.51% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.33:50010 (worker04.sys76.com) Hostname: worker04.sys76.com Decommission Status : Normal Configured Capacity: 2351875026944 (2.14 TB) DFS Used: 4573270016 (4.26 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2347124950656 (2.13 TB) DFS Used%: 0.19% DFS Remaining%: 99.80% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.31:50010 (worker02.sys76.com) Hostname: worker02.sys76.com Decommission Status : Normal Configured Capacity: 2351875026944 (2.14 TB) DFS Used: 5077798912 (4.73 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2346627408110 (2.13 TB) DFS Used%: 0.22% DFS Remaining%: 99.78% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 6 Last contact: Wed Jan 31 13:34:02 UTC 2018 Name: 10.164.252.30:50010 (worker01.sys76.com) Hostname: worker01.sys76.com Decommission Status : Normal Configured Capacity: 2351875551232 (2.14 TB) DFS Used: 4538212352 (4.23 GB) Non DFS Used: 0 (0 B) DFS Remaining: 2347141642744 (2.13 TB) DFS Used%: 0.19% DFS Remaining%: 99.80% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 8 Last contact: Wed Jan 31 13:34:02 UTC 2018 ... View more

by the following API we can check component status on the relevant master machine, my target is to verify that all componet are stop before reboot the machine , but in case ambari server is down then we cant to use the API my quastion is - what the other alternative to check all componet are really stop on master machine ( in case ambari server is down ) curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://master02.sys65.com:8080/api/v1/clusters/HDP/hosts/master02.sys65.com/host_components?fields=HostRoles/state ... View more