Reply
New Contributor
Posts: 3
Registered: ‎03-12-2018

Re: Fail to start HDFS in Cluster Setup

I have the issue 

 

Creating HDFS /tmp directory

 

Command (Create /tmp Directory (241)) has failed

 

---------

Mon Mar 12 16:22:02 +03 2018
Current working directory: /run/cloudera-scm-agent/process/115-hdfs-NAMENODE-createtmp
Launching one-off process: /usr/lib64/cmf/service/hdfs/hdfs.sh mkdir /tmp hdfs supergroup 1777
Mon Mar 12 16:22:02 +03 2018
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using /usr/java/jdk1.7.0_67-cloudera as JAVA_HOME
using 5 as CDH_VERSION
using /run/cloudera-scm-agent/process/115-hdfs-NAMENODE-createtmp as CONF_DIR
using  as SECURE_USER
using  as SECURE_GROUP
CONF_DIR=/run/cloudera-scm-agent/process/115-hdfs-NAMENODE-createtmp
CMF_CONF_DIR=/etc/cloudera-scm-agent
unlimited
2018-03-12 15:22:04 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Directory /tmp does not currently exist.
2018-03-12 15:22:04 WARN  CreateHdfsDirUtil:377 - Connection failed 1 time(s). Trying again...
Directory /tmp does not currently exist.
2018-03-12 15:22:05 WARN  CreateHdfsDirUtil:377 - Connection failed 2 time(s). Trying again...
Directory /tmp does not currently exist.
2018-03-12 15:22:06 WARN  CreateHdfsDirUtil:377 - Connection failed 3 time(s). Trying again...
Directory /tmp does not currently exist.
2018-03-12 15:22:07 WARN  CreateHdfsDirUtil:377 - Connection failed 4 time(s). Trying again...
Directory /tmp does not currently exist.
2018-03-12 15:22:08 WARN  CreateHdfsDirUtil:377 - Connection failed 5 time(s). Trying again...
Directory /tmp does not currently exist.

-----

There is enough space but gives error:

2:33:51.723 PMWARNNameNodeResourceChecker
Space available on volume '/dev/mapper/centos_node1-root' is 0, which is below the configured reserved amount 104857600
2:33:51.723 PMWARNFSNamesystem
NameNode low on available disk space. Already in safe mode.
2:33:51.724 PMINFOStateChange
STATE* Safe mode is ONResources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
2:33:53.138 PMWARNUserGroupInformation
PriviledgedActionException as:hdfs (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /tmp/.cloudera_health_monitoring_canary_files. Name node is in safe mode.
Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
2:33:53.139 PMINFOServer
IPC Server handler 8 on 8020, call org.apache.hadoop.hdfs.protocol.ClientProtocol.mkdirs from 10.0.0.1:52192 Call#447 Retry#0: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /tmp/.cloudera_health_monitoring_canary_files. Name node is in safe mode.
Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
Expert Contributor
Posts: 133
Registered: ‎01-08-2018

Re: Fail to start HDFS in Cluster Setup

According to ther error you posted, your NameNode is in Safe Mode. There should be an Active Name Node in order to perform any HDFS action.

Announcements