Reply
New Contributor
Posts: 3
Registered: ‎06-27-2017

Datanode failing to start as root user

I am seeing the below error while trying to start a datanode as root user, where as the script is trying to write the PID and log file as hdfs user.

 

Getting below error while starting datanode

 

[root@testqoa lib]# /etc/init.d/hadoop-hdfs-datanode start

chown: changing ownership of /var/log/hadoop-hdfs': Operation not permitted`
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-
testqoa.out
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 159: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`
`head: cannot open /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out'
for reading: No such file or directory`

/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 177: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`

/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 178: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`
Failed to start Hadoop datanode. Return value: 1 [FAILED]`
[root@testqoa hadoop-hdfs]# `

 

Tried executing the below command before the datanode restart but error s still the same

 

[root@testqoa run]# chown -R root:root /var/run/hadoop-hdfs
[root@testqoa run]# rm -rf /var/run/hadoop-hdfs/*
[root@testqoa run]# chmod -R 755 /var/run/hadoop-hdfs
[root@testqoa run]# chown -R root:root /data
[root@testqoa run]# chmod -R 755 /data
[root@testqoa hadoop-hdfs]# chown -R root:root /var/lib/hadoop-hdfs`


"hdfs-site.xml" datanode config

`<property>`

`<name>dfs.datanode.data.dir</name>`
`

file:///data/1/dfs/dn,file:///data/2/dfs/dn,file:///data/3/dfs/dn,file:///data/4/dfs/dn`

</property>

hadoop-hdfs-datanode script

DAEMON="hadoop-hdfs-datanode"
DESC="Hadoop datanode"
EXEC_PATH="/usr/lib/hadoop/sbin/hadoop-daemon.sh"
EXEC_DIR=""
SVC_USER="root"
DAEMON_FLAGS="datanode"
CONF_DIR="/etc/hadoop/conf"
PIDFILE="/var/run/hadoop-hdfs/hadoop-hdfs-datanode.pid"
LOCKDIR="/var/lock/subsys"
LOCKFILE="$LOCKDIR/hadoop-hdfs-datanode"
WORKING_DIR="/var/lib/hadoop-hdfs"`
No logs got generated after the start

[root@testqoa hadoop-hdfs]# ls -lt /var/log/hadoop-hdfs
total 0
[root@testqoa hadoop-hdfs]#

PID got started using hdfs user where as i started using root user and changed SVC_USER="root"

[root@testqoa hadoop-hdfs]# ls -lt /var/run/hadoop-hdfs
total 4
-rw-r--r-- 1 hdfs hadoop 6 Jun 26 12:33 hadoop-hdfs-datanode.pid

Champion
Posts: 777
Registered: ‎05-16-2016

Re: Datanode failing to start as root user

[ Edited ]

I hope you are using CDH

trying runining  the below

 

sudo service hadoop-hdfs-datanode start 
sudo service hadoop-hdfs-namenode start
sudo service hadoop-hdfs-secondarynamenode start 

then replace start with status / restart . 

create a normal user , dont use root

add the user to group hadoop

New Contributor
Posts: 3
Registered: ‎06-27-2017

Re: Datanode failing to start as root user

Thanks for your reply. Even though i have root acess i have limited control to add the user to suoder or add them to hdfs group. Is it possible to run the service as root even though its not a best practises to be followed. I am trying this out in my dev env. I have even modified the SVC_USER="ROOT" 

[root@testqoa spnair]# sudo service hadoop-hdfs-datanode start
sudo: service: command not found
[root@testqoa spnair]# sudo /etc/init.d/hadoop-hdfs-datanode start

chown: changing ownership of `/var/log/hadoop-hdfs': Operation not permitted
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 159: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied
 
head: cannot open `/var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out' for reading: No such file or directory
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 177: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 178: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied
Failed to start Hadoop datanode. Return value: 1 [FAILED]

 

Champion
Posts: 777
Registered: ‎05-16-2016

Re: Datanode failing to start as root user

Although I havent tried , trying putting 777 on the chmod to that directory and run it as root. 

 

chmod -R 777 /etc/init.d/

it is very very bad practice though.