Community Articles
Find and share helpful community-sourced technical articles.
Labels (1)
Cloudera Employee

Being a secured cluster, the data transfer protocol of DataNode does not use the RPC framework of Hadoop, DataNode must authenticate itself by using privileged ports which are specified by dfs.datanode.address and dfs.datanode.http.address. This authentication is based on the assumption that the attacker won’t be able to get root privileges.

When you execute hdfs datanode command as root, server process binds privileged port at first, then drops privilege and runs as the user account specified by HADOOP_SECURE_DN_USER.

You can identify that it is a kerberized cluster from the ps -ef output, in the end it says SecureDatanode.

"XX:+PrintGCDateStamps -Xms2048m -Xmx2048m,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT,RFAS org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter"

ps -ef|grep datanode ( have cut short the output for space purpose)

root 2454 1 0 Jul27 ? 00:00:00 jsvc.exec -Dproc_datanode -outfile /var/log/hadoop/hdfs/jsvc.out -errfile /var/log/hadoop/hdfs/jsvc.err -pidfile /var/run/hadoop/hdfs/ -nodetach -user hdfs -cp /usr/hdp/current/hadoop-client/conf:/usr/hdp/*:/usr/hdp/*:/usr/hdp/*:/usr/hdp/*:/usr/hdp/*:/usr/hdp/ -Xmx2048m,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT,RFAS org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter

hdfs 2486 2454 0 Jul27 ? 01:40:11 jsvc.exec -Dproc_datanode -outfile /var/log/hadoop/hdfs/jsvc.out -errfile /var/log/hadoop/hdfs/jsvc.err -pidfile /var/run/hadoop/hdfs/ -nodetach -user hdfs -cp /usr/hdp/current/hadoop- -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xms2048m -Xmx2048m,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT,RFAS org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter

0 Kudos
Cloudera Employee

Hi Bala,

Why this secure process is for DN only ? No as such process runs for NN.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.
Version history
Last update:
‎03-23-2017 07:20 PM
Updated by:
Top Kudoed Authors
; ;