Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Cloudera Employee

ISSUE : Did OS Patch on all their hosts on the cluster and after that Datanodes are started but none of them are alive in the HDFS Dashboard and all the blocks are showing missing

Linux cl-master1 2.6.32-696.3.2.el6.x86_64 #1 SMP Wed Jun 7 11:51:39 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

RESOLUTION :

This is a Kernel Bug https://issues.apache.org/jira/browse/HDFS-12029

Please put the Namenode in Safe mode using the below command

Hdfs dfsadmin –safemode enter

Then add “-Xss2m” on in between the below lines

/usr/hdp/2.6.0.3-8/hadoop-hdfs/bin/hdfs.distro on all the Datanodes

exec "$JSVC" \
-Xss2m \
-Dproc_$COMMAND -outfile "$JSVC_OUTFILE" \
           -errfile "$JSVC_ERRFILE" \
           -pidfile "$HADOOP_SECURE_DN_PID" \
           -nodetach \
           -user "$HADOOP_SECURE_DN_USER" \
            -cp "$CLASSPATH" \
           $JAVA_HEAP_MAX $HADOOP_OPTS \
           org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter "$@"

Please restart the data nodes from Ambari and leave the safe mode on Namenode.

hdfs dfsadmin -leave safemode

394 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎06-29-2017 07:04 PM
Updated by:
 
Contributors
Top Kudoed Authors