Created 04-29-2016 09:57 PM
I have started the nodemanager and resourcemanager via Yarn but still its not showing me the node list.
all the services are up and running in Ambari-Horntonworks
nohup yarn resourcemanager &
nohup yarn nodemanager &
yarn node -list
16/04/29 17:59:22 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 Total Nodes:0 Node-Id Node-State Node-Http-Address Number-of-Running-Containers [hdfs@ip-172-31-11-183 ~]$
Created 05-02-2016 05:54 PM
ah dumb me . I had choosen secondaryMaster node to run yarn , but I was giving yarn node list command on the Master.
from SecondaryMaster node the command is working fine
[yarn@ip-172-31-11-180 ~]$ yarn node -list 16/05/02 13:55:02 INFO impl.TimelineClientImpl: Timeline service address: http://ip-172-31-11-180.ec2.internal:8188/ws/v1/timeline/ 16/05/02 13:55:02 INFO client.RMProxy: Connecting to ResourceManager at ip-172-31-11-180.ec2.internal/172.31.11.180:8050 Total Nodes:3 Node-Id Node-State Node-Http-Address Number-of-Running-Containers ip-172-31-11-179.ec2.internal:45454 RUNNING ip-172-31-11-179.ec2.internal:8042 0 ip-172-31-11-181.ec2.internal:45454 RUNNING ip-172-31-11-181.ec2.internal:8042 0 ip-172-31-11-182.ec2.internal:45454 RUNNING ip-172-31-11-182.ec2.internal:8042 0
Created 04-29-2016 10:04 PM
Check if the nodemanager service is running on the nodes that you are expecting.
ps -ef | grep nodemanager
should show if it is running. If it is running, take a look to see if resourcemanager is running.
ps -ef | grep resourcemanager
on resourcemanager node. Please also paste resourcemanager and nodemanager logs (from /var/log/hadoop-yarn/yarn folder on those nodes)
Created 05-02-2016 02:44 PM
the node manager and resource manager are running , please see below :
[hdfs@ip-172-31-11-183 ~]$ ps -ef | grep nodemanager hdfs 5549 2768 0 10:43 pts/0 00:00:00 /bin/sh ./nodemanager.sh hdfs 5550 5549 12 10:43 pts/0 00:00:10 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_nodemanager -Xmx1000m -Dhdp.version=2.3.4.0-3485 -server -Dhadoop.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dyarn.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dhadoop.log.file=yarn.log -Dyarn.log.file=yarn.log -Dyarn.home.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn -Dhadoop.home.dir=/usr/hdp/2.3.4.0-3485/hadoop -Dhadoop.root.logger=INFO,console -Dyarn.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native -classpath /usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/.//*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/./:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/.//*:::/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/conf/nm-config/log4j.properties org.apache.hadoop.yarn.server.nodemanager.NodeManager hdfs 6902 2768 0 10:45 pts/0 00:00:00 grep nodemanager [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ ps -ef | grep resourcemanager hdfs 5685 2768 0 10:43 pts/0 00:00:00 /bin/sh ./resourcemanager.sh hdfs 5686 5685 13 10:43 pts/0 00:00:11 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_resourcemanager -Xmx1000m -Dhdp.version=2.3.4.0-3485 -Dyarn.server.resourcemanager.appsummary.logger=INFO,RMSUMMARY -Dhadoop.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dyarn.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dhadoop.log.file=yarn.log -Dyarn.log.file=yarn.log -Dyarn.home.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn -Dhadoop.home.dir=/usr/hdp/2.3.4.0-3485/hadoop -Dhadoop.root.logger=INFO,console -Dyarn.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native -classpath /usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/.//*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/./:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/.//*:::/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/conf/rm-config/log4j.properties org.apache.hadoop.yarn.server.resourcemanager.ResourceManager hdfs 6908 2768 0 10:45 pts/0 00:00:00 grep resourcemanager [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ yarn node -list 16/05/02 10:45:27 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 Total Nodes:0 Node-Id Node-State Node-Http-Address Number-of-Running-Containers [hdfs@ip-172-31-11-183 ~]$
Created 05-02-2016 02:46 PM
also there are no files in the yarn directory :
[hdfs@ip-172-31-11-183 yarn]$ pwd /var/log/hadoop/yarn [hdfs@ip-172-31-11-183 yarn]$ ls [hdfs@ip-172-31-11-183 yarn]$
Created 05-02-2016 05:35 PM
from the web dash board it works though so resourcemanager and nodemanagers must be up .
wondering why its not working from command prompt , how can I debug ?
Created 05-02-2016 02:59 PM
sorry I added my comments as answers, if admins are reading this can you please delete my two answers ?
the node manager and resource manager are running , please see below :
[hdfs@ip-172-31-11-183 ~]$ ps -ef | grep nodemanager hdfs 5549 2768 0 10:43 pts/0 00:00:00 /bin/sh ./nodemanager.sh hdfs 5550 5549 12 10:43 pts/0 00:00:10 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_nodemanager -Xmx1000m -Dhdp.version=2.3.4.0-3485 -server -Dhadoop.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dyarn.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dhadoop.log.file=yarn.log -Dyarn.log.file=yarn.log -Dyarn.home.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn -Dhadoop.home.dir=/usr/hdp/2.3.4.0-3485/hadoop -Dhadoop.root.logger=INFO,console -Dyarn.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native -classpath /usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/.//*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/./:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/.//*:::/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/conf/nm-config/log4j.properties org.apache.hadoop.yarn.server.nodemanager.NodeManager hdfs 6902 2768 0 10:45 pts/0 00:00:00 grep nodemanager [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ ps -ef | grep resourcemanager hdfs 5685 2768 0 10:43 pts/0 00:00:00 /bin/sh ./resourcemanager.sh hdfs 5686 5685 13 10:43 pts/0 00:00:11 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_resourcemanager -Xmx1000m -Dhdp.version=2.3.4.0-3485 -Dyarn.server.resourcemanager.appsummary.logger=INFO,RMSUMMARY -Dhadoop.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dyarn.log.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn/logs -Dhadoop.log.file=yarn.log -Dyarn.log.file=yarn.log -Dyarn.home.dir=/usr/hdp/2.3.4.0-3485/hadoop-yarn -Dhadoop.home.dir=/usr/hdp/2.3.4.0-3485/hadoop -Dhadoop.root.logger=INFO,console -Dyarn.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native -classpath /usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/conf:/usr/hdp/2.3.4.0-3485/hadoop/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/.//*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/./:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-hdfs/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/lib/*:/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/.//*:::/usr/hdp/2.3.4.0-3485/hadoop-yarn/.//*:/usr/hdp/2.3.4.0-3485/hadoop-yarn/lib/*:/usr/hdp/2.3.4.0-3485/hadoop/conf/rm-config/log4j.properties org.apache.hadoop.yarn.server.resourcemanager.ResourceManager hdfs 6908 2768 0 10:45 pts/0 00:00:00 grep resourcemanager [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ [hdfs@ip-172-31-11-183 ~]$ yarn node -list 16/05/02 10:45:27 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 Total Nodes:0 Node-Id Node-State Node-Http-Address Number-of-Running-Containers [hdfs@ip-172-31-11-183 ~]$
Created 05-02-2016 05:44 PM
So, I see both Nodemanager and resourcemanager running on that host.
Can you paste the screenshot from resourcemanager as well.. http://resourcemanagerhost:8088/cluster
This should show how many nodemanagers have registered with your resourcemanager. If that shows 0 hosts as well, attach and /var/log/hadoop-yarn/yarn/yarn-yarn-nodemanager-<hostname>.log and /var/log/hadoop-yarn/yarn/yarn-yarn-resorucemanager-<hostname>.log
Created 05-02-2016 05:54 PM
ah dumb me . I had choosen secondaryMaster node to run yarn , but I was giving yarn node list command on the Master.
from SecondaryMaster node the command is working fine
[yarn@ip-172-31-11-180 ~]$ yarn node -list 16/05/02 13:55:02 INFO impl.TimelineClientImpl: Timeline service address: http://ip-172-31-11-180.ec2.internal:8188/ws/v1/timeline/ 16/05/02 13:55:02 INFO client.RMProxy: Connecting to ResourceManager at ip-172-31-11-180.ec2.internal/172.31.11.180:8050 Total Nodes:3 Node-Id Node-State Node-Http-Address Number-of-Running-Containers ip-172-31-11-179.ec2.internal:45454 RUNNING ip-172-31-11-179.ec2.internal:8042 0 ip-172-31-11-181.ec2.internal:45454 RUNNING ip-172-31-11-181.ec2.internal:8042 0 ip-172-31-11-182.ec2.internal:45454 RUNNING ip-172-31-11-182.ec2.internal:8042 0