Can someone guide me on why YARN is utilizing zero out of 16.5 ? If this is an error how can I fix this ?
You can get detailed explaination about Yarn "Cluster Memory" in the following thread.
Regaarding 0 (Zero Memory utiliaztion). Until you run a job you wont see the memory being utilized.
So please try running some job like following and then refresh the ambari Yarn dashboard.
# su - spark
# export SPARK_MAJOR_VERSION=2
# cd /usr/hdp/current/spark2-client
# ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn-client --num-executors 3 --driver-memory 512m --executor-memory 512m --executor-cores 1 examples/jars/spark-examples*.jar 10
View solution in original post