Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Datanode failing to start as root user

Highlighted

Datanode failing to start as root user

New Contributor

I am seeing the below error while trying to start a datanode as root user, where as the script is trying to write the PID and log file as hdfs user.

 

Getting below error while starting datanode

 

[root@testqoa lib]# /etc/init.d/hadoop-hdfs-datanode start

chown: changing ownership of /var/log/hadoop-hdfs': Operation not permitted`
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-
testqoa.out
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 159: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`
`head: cannot open /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out'
for reading: No such file or directory`

/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 177: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`

/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 178: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`
Failed to start Hadoop datanode. Return value: 1 [FAILED]`
[root@testqoa hadoop-hdfs]# `

 

Tried executing the below command before the datanode restart but error s still the same

 

[root@testqoa run]# chown -R root:root /var/run/hadoop-hdfs
[root@testqoa run]# rm -rf /var/run/hadoop-hdfs/*
[root@testqoa run]# chmod -R 755 /var/run/hadoop-hdfs
[root@testqoa run]# chown -R root:root /data
[root@testqoa run]# chmod -R 755 /data
[root@testqoa hadoop-hdfs]# chown -R root:root /var/lib/hadoop-hdfs`


"hdfs-site.xml" datanode config

`<property>`

`<name>dfs.datanode.data.dir</name>`
`

file:///data/1/dfs/dn,file:///data/2/dfs/dn,file:///data/3/dfs/dn,file:///data/4/dfs/dn`

</property>

hadoop-hdfs-datanode script

DAEMON="hadoop-hdfs-datanode"
DESC="Hadoop datanode"
EXEC_PATH="/usr/lib/hadoop/sbin/hadoop-daemon.sh"
EXEC_DIR=""
SVC_USER="root"
DAEMON_FLAGS="datanode"
CONF_DIR="/etc/hadoop/conf"
PIDFILE="/var/run/hadoop-hdfs/hadoop-hdfs-datanode.pid"
LOCKDIR="/var/lock/subsys"
LOCKFILE="$LOCKDIR/hadoop-hdfs-datanode"
WORKING_DIR="/var/lib/hadoop-hdfs"`
No logs got generated after the start

[root@testqoa hadoop-hdfs]# ls -lt /var/log/hadoop-hdfs
total 0
[root@testqoa hadoop-hdfs]#

PID got started using hdfs user where as i started using root user and changed SVC_USER="root"

[root@testqoa hadoop-hdfs]# ls -lt /var/run/hadoop-hdfs
total 4
-rw-r--r-- 1 hdfs hadoop 6 Jun 26 12:33 hadoop-hdfs-datanode.pid

3 REPLIES 3

Re: Datanode failing to start as root user

Champion

I hope you are using CDH

trying runining  the below

 

sudo service hadoop-hdfs-datanode start 
sudo service hadoop-hdfs-namenode start
sudo service hadoop-hdfs-secondarynamenode start 

then replace start with status / restart . 

create a normal user , dont use root

add the user to group hadoop

Re: Datanode failing to start as root user

New Contributor

Thanks for your reply. Even though i have root acess i have limited control to add the user to suoder or add them to hdfs group. Is it possible to run the service as root even though its not a best practises to be followed. I am trying this out in my dev env. I have even modified the SVC_USER="ROOT" 

[root@testqoa spnair]# sudo service hadoop-hdfs-datanode start
sudo: service: command not found
[root@testqoa spnair]# sudo /etc/init.d/hadoop-hdfs-datanode start

chown: changing ownership of `/var/log/hadoop-hdfs': Operation not permitted
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 159: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied
 
head: cannot open `/var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out' for reading: No such file or directory
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 177: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 178: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied
Failed to start Hadoop datanode. Return value: 1 [FAILED]

 

Re: Datanode failing to start as root user

Champion

Although I havent tried , trying putting 777 on the chmod to that directory and run it as root. 

 

chmod -R 777 /etc/init.d/

it is very very bad practice though.