Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

DataNodes status not consistent

Solved Go to solution
Highlighted

DataNodes status not consistent

Hi All,

I have a cluster with 4 machines. In each machine has been installed a DataNode. I reported below the screenshots showing the Ambari status.

It is correctly showing 4/4 datanodes however only 1 seems live.

My questions are: *) why is not showing 4 lives datanodes? *) is this affecting also that block are not replicated (see "under replicated blocks") *) also, when running a spark job i get:

YarnSchedulerBackend$YarnSchedulerEndpoint: Container marked as failed: container_e07_1480428595380_0003_02_000003 on host: slave01.hortonworks.com. Exit status: -1000. Diagnostics: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1459687468-127.0.1.1-1479480481481:blk_1073741831_1007 file=/hdp/apps/2.5.0.0-1245/spark/spark-hdp-assembly.jar

9878-02.png

9879-01.png

Could you please help me on the above issues?

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: DataNodes status not consistent

Hi All,

i solved the issue adding the following configuration in the "Custom hdfs-site" section.

<property>
  <name>dfs.namenode.rpc-bind-host</name>
  <value>0.0.0.0</value>
</property>

I modified also the following in the "Advanced hdfs-site" section:

from nameMyServer:8020

to ipMyServer:8020

Regards

Alessandro

View solution in original post

2 REPLIES 2
Highlighted

Re: DataNodes status not consistent

Please check NN UI, if all DNs are active over there. If there are servers marked dead, we will need to see the DN logs on each of these nodes.

Highlighted

Re: DataNodes status not consistent

Hi All,

i solved the issue adding the following configuration in the "Custom hdfs-site" section.

<property>
  <name>dfs.namenode.rpc-bind-host</name>
  <value>0.0.0.0</value>
</property>

I modified also the following in the "Advanced hdfs-site" section:

from nameMyServer:8020

to ipMyServer:8020

Regards

Alessandro

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here