Created on 06-27-2017 04:39 AM - edited 09-16-2022 04:50 AM
I am seeing the below error while trying to start a datanode as root user, where as the script is trying to write the PID and log file as hdfs user.
Getting below error while starting datanode
[root@testqoa lib]# /etc/init.d/hadoop-hdfs-datanode start
chown: changing ownership of /var/log/hadoop-hdfs': Operation not permitted`
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-
testqoa.out
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 159: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`
`head: cannot open /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out'
for reading: No such file or directory`
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 177: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`
/usr/lib/hadoop/sbin/hadoop-daemon.sh: line 178: /var/log/hadoop-
hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied`
Failed to start Hadoop datanode. Return value: 1 [FAILED]`
[root@testqoa hadoop-hdfs]# `
Tried executing the below command before the datanode restart but error s still the same
[root@testqoa run]# chown -R root:root /var/run/hadoop-hdfs
[root@testqoa run]# rm -rf /var/run/hadoop-hdfs/*
[root@testqoa run]# chmod -R 755 /var/run/hadoop-hdfs
[root@testqoa run]# chown -R root:root /data
[root@testqoa run]# chmod -R 755 /data
[root@testqoa hadoop-hdfs]# chown -R root:root /var/lib/hadoop-hdfs`
"hdfs-site.xml" datanode config
`<property>`
`<name>dfs.datanode.data.dir</name>`
`
file:///data/1/dfs/dn,file:///data/2/dfs/dn,file:///data/3/dfs/dn,file:///data/4/dfs/dn`
</property>
hadoop-hdfs-datanode script
DAEMON="hadoop-hdfs-datanode"
DESC="Hadoop datanode"
EXEC_PATH="/usr/lib/hadoop/sbin/hadoop-daemon.sh"
EXEC_DIR=""
SVC_USER="root"
DAEMON_FLAGS="datanode"
CONF_DIR="/etc/hadoop/conf"
PIDFILE="/var/run/hadoop-hdfs/hadoop-hdfs-datanode.pid"
LOCKDIR="/var/lock/subsys"
LOCKFILE="$LOCKDIR/hadoop-hdfs-datanode"
WORKING_DIR="/var/lib/hadoop-hdfs"`
No logs got generated after the start
[root@testqoa hadoop-hdfs]# ls -lt /var/log/hadoop-hdfs
total 0
[root@testqoa hadoop-hdfs]#
PID got started using hdfs user where as i started using root user and changed SVC_USER="root"
[root@testqoa hadoop-hdfs]# ls -lt /var/run/hadoop-hdfs
total 4
-rw-r--r-- 1 hdfs hadoop 6 Jun 26 12:33 hadoop-hdfs-datanode.pid
Created on 06-27-2017 06:00 AM - edited 06-27-2017 06:01 AM
I hope you are using CDH
trying runining the below
sudo service hadoop-hdfs-datanode start sudo service hadoop-hdfs-namenode start sudo service hadoop-hdfs-secondarynamenode start then replace start with status / restart .
create a normal user , dont use root
add the user to group hadoop
Created 06-27-2017 06:42 AM
Thanks for your reply. Even though i have root acess i have limited control to add the user to suoder or add them to hdfs group. Is it possible to run the service as root even though its not a best practises to be followed. I am trying this out in my dev env. I have even modified the SVC_USER="ROOT"
[root@testqoa spnair]# sudo service hadoop-hdfs-datanode start sudo: service: command not found [root@testqoa spnair]# sudo /etc/init.d/hadoop-hdfs-datanode start chown: changing ownership of `/var/log/hadoop-hdfs': Operation not permitted starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out /usr/lib/hadoop/sbin/hadoop-daemon.sh: line 159: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied head: cannot open `/var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out' for reading: No such file or directory /usr/lib/hadoop/sbin/hadoop-daemon.sh: line 177: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied /usr/lib/hadoop/sbin/hadoop-daemon.sh: line 178: /var/log/hadoop-hdfs/hadoop-hdfs-datanode-testqoa.out: Permission denied Failed to start Hadoop datanode. Return value: 1 [FAILED]
Created 06-27-2017 08:10 AM
Although I havent tried , trying putting 777 on the chmod to that directory and run it as root.
chmod -R 777 /etc/init.d/
it is very very bad practice though.