Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to bring back a slave node that is lost

How to bring back a slave node that is lost

Explorer

Hi,

I don't have access to network provider now but I wonder if there's a way to bring back a slave node that is lost? in 8088 it shows the node is lost but in 50070 it says the node is healthy. I can ssh all the slave VMs and hadoop dfsadmin report shows all the nodes are healthy so I don't know why it tells node is lost in 8088.

P.S.: Should I restart yarn resource manager for this purpose and how?

P.P.S.: Why the container shows a -1 and how can I fix it in 8088?

889-50070.png

888-8088.png

5 REPLIES 5
Highlighted

Re: How to bring back a slave node that is lost

Super Guru

@Mona Jalal - It looks like your datanode service is running fine but there seems problem with nodemanager, login to lost not and try to look at nodemanagers logs ( /var/log/hadoop-yarn/yarn/yarn-yarn-nodemanager-<hostname>.log )

You can restart nodemanager service on lost node to get this back in action. No need to restart Resource manager.

Highlighted

Re: How to bring back a slave node that is lost

Explorer

Hey, do you know which -site.xml file defines the location of NodeManager logs?

Highlighted

Re: How to bring back a slave node that is lost

Super Guru

As per screenshot given, it looks like node is decommissioned.

Goto ambari UI --> Click on Yarn service --> Click on Nodemanagers --> Select lost node --> Goto Actions (top left) --> Filtered host --> Nodemanagers --> Recommission

screen-shot-2015-12-17-at-85641-am.png

Highlighted

Re: How to bring back a slave node that is lost

Explorer

I am not the network provider and I don't know what Ambari is. Our TA gave us these VMs from CloudLab.us so can I still do this and if yes, which port of public IP should I browse to?

Highlighted

Re: How to bring back a slave node that is lost

Expert Contributor

@Mona Jalal

Default port for ambari is 8080

Don't have an account?
Coming from Hortonworks? Activate your account here