Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Adding datanode only partially works

Highlighted

Adding datanode only partially works

Explorer

I recently added two datanodes to an existing cluster through amabri. The datanodes installed and were added to the UI, but through the hadoop management page it still only shows 12 live datanodes. Also, no new space was added to hdfs. /etc/hadoop/conf/slaves only shows the original 12 nodes.

2 REPLIES 2
Highlighted

Re: Adding datanode only partially works

@Alex Eifler

Each slave node in Yet Another Resource Negotiator (YARN) has a Node Manager daemon, which acts as a slave for the Resource Manager. As with the TaskTracker, each slave node has a service that ties it to the processing service (Node Manager) and the storage service (DataNode) that enable Hadoop to be a distributed system.

I'm not sure how you added the data node, but please be advised that you need also to add the node manager service running on that host with data node. This is a common mistake and I get it. You get easily confused by adding only "add data" service. Add the node manager service too, restart the node then do hdfs rebalance. Another common mistake on not doing it. Drives also need to be prepared properly to be used.

+++

If this helped, please vote/accept answer.

Re: Adding datanode only partially works

Explorer

Thanks for the info Constantin. I was adding the node through ambari using the UI. Turns out the datanode will install and run without being able to talk to the namenode. Once I addressed the issue with connectivity everything started to work as expected.

Don't have an account?
Coming from Hortonworks? Activate your account here