YARN command not working

[#1]

I have started the nodemanager and resourcemanager via Yarn but still its not showing me the node list.

all the services are up and running in Ambari-Horntonworks

nohup yarn resourcemanager &

nohup yarn nodemanager &

yarn node -list

16/04/29 17:59:22 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 Total Nodes:0 Node-Id Node-State Node-Http-Address Number-of-Running-Containers [hdfs@ip-172-31-11-183 ~]$

[#2]

Check if the nodemanager service is running on the nodes that you are expecting.

ps -ef | grep nodemanager 

should show if it is running. If it is running, take a look to see if resourcemanager is running.

ps -ef | grep resourcemanager

on resourcemanager node. Please also paste resourcemanager and nodemanager logs (from /var/log/hadoop-yarn/yarn folder on those nodes)

[#3]

the node manager and resource manager are running , please see below :

[hdfs@ip-172-31-11-183 ~]$ ps -ef | grep nodemanager hdfs 5549 2768 0 10:43 pts/0 00:00:00 /bin/sh ./nodemanager.sh hdfs 5550 5549 12 10:43 pts/0 00:00:10 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_nodemanager -Xmx1000m -Dhdp.version=2.3.4.0-3485 -server -Dhadoop.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dyarn.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dhadoop.log.file=yarn.log -Dyarn.log.file=yarn.log -Dyarn.home.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn -Dhadoop.home.dir=/usr/hdp/2.3.4.0-3485/hadoop -Dhadoop.root.logger=INFO,console -Dyarn.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native -classpath /usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/.//*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/./:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/.//*:::/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/conf/nm-config/log4j.properties org.apache.hadoop.yarn.server.nodemanager.NodeManager hdfs 6902 2768 0 10:45 pts/0 00:00:00 grep nodemanager [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ ps -ef | grep resourcemanager hdfs 5685 2768 0 10:43 pts/0 00:00:00 /bin/sh ./resourcemanager.sh hdfs 5686 5685 13 10:43 pts/0 00:00:11 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_resourcemanager -Xmx1000m -Dhdp.version=2.3.4.0-3485 -Dyarn.server.resourcemanager.appsummary.logger=INFO,RMSUMMARY -Dhadoop.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dyarn.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dhadoop.log.file=yarn.log -Dyarn.log.file=yarn.log -Dyarn.home.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn -Dhadoop.home.dir=/usr/hdp/2.3.4.0-3485/hadoop -Dhadoop.root.logger=INFO,console -Dyarn.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native -classpath /usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/.//*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/./:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/.//*:::/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/conf/rm-config/log4j.properties org.apache.hadoop.yarn.server.resourcemanager.ResourceManager hdfs 6908 2768 0 10:45 pts/0 00:00:00 grep resourcemanager [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ yarn node -list 16/05/02 10:45:27 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 Total Nodes:0 Node-Id Node-State Node-Http-Address Number-of-Running-Containers [hdfs@ip-172-31-11-183 ~]$

[#4]

also there are no files in the yarn directory :

[hdfs@ip-172-31-11-183 yarn]$ pwd /var/log/hadoop/yarn [hdfs@ip-172-31-11-183 yarn]$ ls [hdfs@ip-172-31-11-183 yarn]$

[#5]

from the web dash board it works though so resourcemanager and nodemanagers must be up .

wondering why its not working from command prompt , how can I debug ?

yarnnodelist.jpg

[#6]

sorry I added my comments as answers, if admins are reading this can you please delete my two answers ?

the node manager and resource manager are running , please see below :

[hdfs@ip-172-31-11-183 ~]$ ps -ef | grep nodemanager hdfs 5549 2768 0 10:43 pts/0 00:00:00 /bin/sh ./nodemanager.sh hdfs 5550 5549 12 10:43 pts/0 00:00:10 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_nodemanager -Xmx1000m -Dhdp.version=2.3.4.0-3485 -server -Dhadoop.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dyarn.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dhadoop.log.file=yarn.log -Dyarn.log.file=yarn.log -Dyarn.home.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn -Dhadoop.home.dir=/usr/hdp/2.3.4.0-3485/hadoop -Dhadoop.root.logger=INFO,console -Dyarn.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native -classpath /usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/.//*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/./:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/.//*:::/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/conf/nm-config/log4j.properties org.apache.hadoop.yarn.server.nodemanager.NodeManager hdfs 6902 2768 0 10:45 pts/0 00:00:00 grep nodemanager [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ ps -ef | grep resourcemanager hdfs 5685 2768 0 10:43 pts/0 00:00:00 /bin/sh ./resourcemanager.sh hdfs 5686 5685 13 10:43 pts/0 00:00:11 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_resourcemanager -Xmx1000m -Dhdp.version=2.3.4.0-3485 -Dyarn.server.resourcemanager.appsummary.logger=INFO,RMSUMMARY -Dhadoop.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dyarn.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dhadoop.log.file=yarn.log -Dyarn.log.file=yarn.log -Dyarn.home.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn -Dhadoop.home.dir=/usr/hdp/2.3.4.0-3485/hadoop -Dhadoop.root.logger=INFO,console -Dyarn.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native -classpath /usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/.//*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/./:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/.//*:::/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/conf/rm-config/log4j.properties org.apache.hadoop.yarn.server.resourcemanager.ResourceManager hdfs 6908 2768 0 10:45 pts/0 00:00:00 grep resourcemanager [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ yarn node -list 16/05/02 10:45:27 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 Total Nodes:0 Node-Id Node-State Node-Http-Address Number-of-Running-Containers [hdfs@ip-172-31-11-183 ~]$

[#7]

So, I see both Nodemanager and resourcemanager running on that host.

Can you paste the screenshot from resourcemanager as well.. http://resourcemanagerhost:8088/cluster

This should show how many nodemanagers have registered with your resourcemanager. If that shows 0 hosts as well, attach and /var/log/hadoop-yarn/yarn/yarn-yarn-nodemanager-<hostname>.log and /var/log/hadoop-yarn/yarn/yarn-yarn-resorucemanager-<hostname>.log

[#8 · Accepted solution]

ah dumb me . I had choosen secondaryMaster node to run yarn , but I was giving yarn node list command on the Master.

from SecondaryMaster node the command is working fine

[yarn@ip-172-31-11-180 ~]$ yarn node -list 16/05/02 13:55:02 INFO impl.TimelineClientImpl: Timeline service address: http://ip-172-31-11-180.ec2.internal:8188/ws/v1/timeline/ 16/05/02 13:55:02 INFO client.RMProxy: Connecting to ResourceManager at ip-172-31-11-180.ec2.internal/172.31.11.180:8050 Total Nodes:3 Node-Id Node-State Node-Http-Address Number-of-Running-Containers ip-172-31-11-179.ec2.internal:45454 RUNNING ip-172-31-11-179.ec2.internal:8042 0 ip-172-31-11-181.ec2.internal:45454 RUNNING ip-172-31-11-181.ec2.internal:8042 0 ip-172-31-11-182.ec2.internal:45454 RUNNING ip-172-31-11-182.ec2.internal:8042 0