hi all we saw few scenarios that disks on datanode machine became full 100% because the files - stdout are huge for example /grid/sdb/hadoop/yarn/log/application_151746342014_5807/container_e37_151003535122014_5807_03_000001/stdout from df -h , we can see df -h /grid/sdb Filesystem Size Used Avail Use% Mounted on /dev/sdb 1.8T 1.8T 0T 100% /grid/sdb any suggestion how to avoid this situation that stdout are huge and actually this issue cause stopping the HDFS component on the datanode second: since the PATH of stdout is: /var/log/hadoop-yarn/containers/[application id]/[container id]/stdout is it possible to limit the file size? or do a purging of stdout when file reached the threshold ? ... View more

hi all we have ambari - 2.6.1 and HDP - 2.6.4 version we set the following: spark.history.fs.cleaner.enabled=true spark.history.fs.cleaner.interval=1d spark.history.fs.cleaner.maxAge=1d but this configuration not deleted the old spark history .inprogress files any suggestion how to force the cleanup? ... View more

hi all in YARN Alerts we saw the following critical alarm 1 NodeManager is unhealthy. we have 36 data node machines that include ( DATANODE , metrics monitor , node manager ) since one of the datanode is the problem , then we need to find the problematic machine can we get advice how to find the datanode with this alert? ... View more