@hrishi1dypim Are you run it from impa-shell or Hue? ... View more

@code0404  See how i'm doing this, but i collect the aggregate metrics per pool, you can just use by the application name or the user.   STARTDATE=`date -d " -1 day " +%s%N | cut -b1-13` ENDDATE=`date +%s%N | cut -b1-13` result=`curl -s http://your-yarn-history-server:8088/ws/v1/cluster/apps?finishedTimeBegin=$STARTDATE&finishedTimeEnd=$ENDDATE` echo $result | python -m json.tool | sed 's/["|,]//g' | grep -E "queue|coreSeconds" | awk -v DC="$DC" ' /queue/ { queue = $2 } /vcoreSeconds/ { arr[queue]+=$2 ; } END { for (x in arr) {print DC ".yarn." x ".cpums="arr[x]} } ' echo $result | python -m json.tool | sed 's/["|,]//g' | grep -E "queue|memorySeconds" | awk -v DC="$DC" ' /queue/ { queue = $2 } /memorySeconds/ { arr1[queue]+=$2 ; } END { for (y in arr1) {print DC ".yarn." y ".memorySeconds="arr1[y]} } ' ... View more

@code0404 Do you have spark jobs or all your jobs are mapreduce?   normally summation of vcores and memory seconds are enough. ... View more

You can increase the time series to store more historical data. What i did in my side, i write a script that collect the data daily and populate it to graphana. ... View more

Hi @code0404 Yes, you can use the Yarn histort API   Please see the following documntation:   https://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html ... View more

all looks fine, you can even reduce yarn.nodemanager.resource.memory-mb ... View more

No need to increase the RAM for the server, it will enough to increase the memory for the NN service, 125GB is too much, increasing the memory willn't solve the issue, but it will not overload the NN. ... View more

This is a bug that fixed at 5.10.1 ... View more