Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.
wilsonc
New Contributor

Re: Mounting NFS from Edge Node with Kerberos

by New Contributor in Support Questions
‎07-17-2017 02:15 PM
‎07-17-2017 02:15 PM
@Kuldeep Kulkarni, Thank you for your response. The user running the NFS Gateway appears to be the hdfs user per the following ps dump [root@smhadoop01 ~]# ps aux | grep nfs root 3555 0.0 0.0 112648 960 pts/0 S+ 09:50 0:00 grep --color=auto nfs root 10467 0.0 0.0 10688 884 ? S Jul16 0:00 jsvc.exec -Dproc_nfs3 -outfile /var/log/hadoop/root/nfs3_jsvc.out -errfile /var/log/hadoop/root/nfs3_jsvc.err -pidfile /var/run/hadoop/root/hadoop_privileged_nfs3.pid -nodetach -user hdfs -cp /usr/hdp/current/hadoop-client/conf:/usr/hdp/2.6.1.0-129/hadoop/lib/*:/usr/hdp/2.6.1.0-129/hadoop/.//*:/usr/hdp/2.6.1.0-129/hadoop-hdfs/./:/usr/hdp/2.6.1.0-129/hadoop-hdfs/lib/*:/usr/hdp/2.6.1.0-129/hadoop-hdfs/.//*:/usr/hdp/2.6.1.0-129/hadoop-yarn/lib/*:/usr/hdp/2.6.1.0-129/hadoop-yarn/.//*:/usr/hdp/2.6.1.0-129/hadoop-mapreduce/lib/*:/usr/hdp/2.6.1.0-129/hadoop-mapreduce/.//*::mysql-connector-java.jar:/usr/hdp/2.6.1.0-129/tez/*:/usr/hdp/2.6.1.0-129/tez/lib/*:/usr/hdp/2.6.1.0-129/tez/conf:mysql-connector-java.jar:mysql-connector-java.jar:/usr/hdp/2.6.1.0-129/tez/*:/usr/hdp/2.6.1.0-129/tez/lib/*:/usr/hdp/2.6.1.0-129/tez/conf -Xmx1024m -Dhdp.version=2.6.1.0-129 -Djava.net.preferIPv4Stack=true -Dhdp.version= -Djava.net.preferIPv4Stack=true -Dhdp.version= -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/ -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.6.1.0-129/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.6.1.0-129/hadoop/lib/native/Linux-amd64-64:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.6.1.0-129/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhdp.version=2.6.1.0-129 -Dhadoop.log.dir=/var/log/hadoop/ -Dhadoop.log.file=hadoop-hdfs-nfs3-smhadoop01.na.corning.com.log -Dhadoop.home.dir=/usr/hdp/2.6.1.0-129/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,RFA -Djava.library.path=:/usr/hdp/2.6.1.0-129/hadoop/lib/native/Linux-amd64-64:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.6.1.0-129/hadoop/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.6.1.0-129/hadoop/lib/native/Linux-amd64-64:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.6.1.0-129/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/root -Dhadoop.id.str=hdfs -Xmx1024m -Dhadoop.security.logger=ERROR,DRFAS -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.nfs.nfs3.PrivilegedNfsGatewayStarter hdfs 10498 0.2 0.3 2898548 496400 ? Sl Jul16 2:54 jsvc.exec -Dproc_nfs3 -outfile /var/log/hadoop/root/nfs3_jsvc.out -errfile /var/log/hadoop/root/nfs3_jsvc.err -pidfile /var/run/hadoop/root/hadoop_privileged_nfs3.pid -nodetach -user hdfs -cp /usr/hdp/current/hadoop-client/conf:/usr/hdp/2.6.1.0-129/hadoop/lib/*:/usr/hdp/2.6.1.0-129/hadoop/.//*:/usr/hdp/2.6.1.0-129/hadoop-hdfs/./:/usr/hdp/2.6.1.0-129/hadoop-hdfs/lib/*:/usr/hdp/2.6.1.0-129/hadoop-hdfs/.//*:/usr/hdp/2.6.1.0-129/hadoop-yarn/lib/*:/usr/hdp/2.6.1.0-129/hadoop-yarn/.//*:/usr/hdp/2.6.1.0-129/hadoop-mapreduce/lib/*:/usr/hdp/2.6.1.0-129/hadoop-mapreduce/.//*::mysql-connector-java.jar:/usr/hdp/2.6.1.0-129/tez/*:/usr/hdp/2.6.1.0-129/tez/lib/*:/usr/hdp/2.6.1.0-129/tez/conf:mysql-connector-java.jar:mysql-connector-java.jar:/usr/hdp/2.6.1.0-129/tez/*:/usr/hdp/2.6.1.0-129/tez/lib/*:/usr/hdp/2.6.1.0-129/tez/conf -Xmx1024m -Dhdp.version=2.6.1.0-129 -Djava.net.preferIPv4Stack=true -Dhdp.version= -Djava.net.preferIPv4Stack=true -Dhdp.version= -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/ -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.6.1.0-129/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.6.1.0-129/hadoop/lib/native/Linux-amd64-64:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.6.1.0-129/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhdp.version=2.6.1.0-129 -Dhadoop.log.dir=/var/log/hadoop/ -Dhadoop.log.file=hadoop-hdfs-nfs3-smhadoop01.na.corning.com.log -Dhadoop.home.dir=/usr/hdp/2.6.1.0-129/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,RFA -Djava.library.path=:/usr/hdp/2.6.1.0-129/hadoop/lib/native/Linux-amd64-64:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.6.1.0-129/hadoop/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.6.1.0-129/hadoop/lib/native/Linux-amd64-64:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.6.1.0-129/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/root -Dhadoop.id.str=hdfs -Xmx1024m -Dhadoop.security.logger=ERROR,DRFAS -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.nfs.nfs3.PrivilegedNfsGatewayStarter And the core-site.xml settings seem to reflect the values you are looking for [root@smhadoop01 ~]# grep -B1 -A2 proxyuser\.hdfs /etc/hadoop/conf/core-site.xml <property> <name>hadoop.proxyuser.hdfs.groups</name> <value>*</value> </property> -- <property> <name>hadoop.proxyuser.hdfs.hosts</name> <value>*</value> </property> And I am able to mount the NFS export on the edge node if I don't enable Kerberos security [root@smhadoop-edge ~]# mount smhadoop01:/ /mnt [root@smhadoop-edge ~]# mount | grep smhadoop01 smhadoop01:/ on /mnt type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.180.104.161,mountvers=3,mountport=4242,mountproto=udp,local_lock=none,addr=10.180.104.161) [root@smhadoop-edge ~]# umount /mnt [root@smhadoop-edge ~]# mount -v -o sec=krb5,noatime,nolock smhadoop01:/ /mnt mount.nfs: timeout set for Mon Jul 17 09:47:26 2017 mount.nfs: trying text-based options 'sec=krb5,nolock,vers=4,addr=10.180.104.161,clientaddr=10.180.104.38' mount.nfs: mount(2): Protocol not supported mount.nfs: trying text-based options 'sec=krb5,nolock,addr=10.180.104.161' mount.nfs: prog 100003, trying vers=3, prot=6 mount.nfs: trying 10.180.104.161 prog 100003 vers 3 prot TCP port 2049 mount.nfs: prog 100005, trying vers=3, prot=17 mount.nfs: trying 10.180.104.161 prog 100005 vers 3 prot UDP port 4242 mount.nfs: mount(2): Permission denied mount.nfs: access denied by server while mounting smhadoop01:/ [root@smhadoop-edge ~]# This really appears to be a problem negotiating Kerberos security on the share. ... View more

Re: Mounting NFS from Edge Node with Kerberos

by New Contributor in Support Questions
‎07-17-2017 02:14 PM
‎07-17-2017 02:14 PM
Sorry, I am still working out this forums mechanics and I posted an answer when I meant to post a Reply. ... View more

Mounting NFS from Edge Node with Kerberos

by New Contributor in Support Questions
‎07-16-2017 05:06 PM
‎07-16-2017 05:06 PM
I am getting an error similar to the one asked about in the question "Problems with Kerberos (NFS and File View)" except my File View and Hadoop fs commands work fine with Kerberos. I have used the Kerberos Wizard and checks completed with no errors. I double checked the settings of the NFS Gateway to ensure it was correctly configured by the Wizard and it matched the instructions for the version of HDP I am running (2.6.1). Here is what I get from the edge node when attempting a mount [root@smhadoop-edge ~]# klist klist: No credentials cache found (filename: /tmp/krb5cc_0) [root@smhadoop-edge ~]# kinit WilsonC Password for WilsonC@NA.CORNING.COM: [root@smhadoop-edge ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: WilsonC@NA.CORNING.COM Valid starting Expires Service principal 07/16/2017 12:59:11 07/16/2017 22:59:11 krbtgt/NA.CORNING.COM@NA.CORNING.COM renew until 07/16/2017 22:59:11 [root@smhadoop-edge ~]# mount -vvv -o nolock,sec=krb5,noatime smhadoop01:/ /mnt mount.nfs: timeout set for Sun Jul 16 13:01:39 2017 mount.nfs: trying text-based options 'nolock,sec=krb5,vers=4,addr=10.180.104.161,clientaddr=10.180.104.38' mount.nfs: mount(2): Protocol not supported mount.nfs: trying text-based options 'nolock,sec=krb5,addr=10.180.104.161' mount.nfs: prog 100003, trying vers=3, prot=6 mount.nfs: trying 10.180.104.161 prog 100003 vers 3 prot TCP port 2049 mount.nfs: prog 100005, trying vers=3, prot=17 mount.nfs: trying 10.180.104.161 prog 100005 vers 3 prot UDP port 4242 mount.nfs: mount(2): Permission denied mount.nfs: access denied by server while mounting smhadoop01:/ However the normal Hadoop commands work fine with Kerberos [root@smhadoop-edge ~]# hadoop fs -mkdir /user/WilsonC/testing [root@smhadoop-edge ~]# hadoop fs -ls /user/WilsonC Found 1 items drwxr-xr-x - WilsonC hdfs 0 2017-07-16 13:02 /user/WilsonC/testing Any help is much appreciated. I am also working on getting Kerberos to work correctly with NFS on Windows if anybody has any pointers. ... View more
Contact Me
Online
Offline
Last Visited
‎07-17-2017 02:14 PM
Public Statistics
Date Registered ‎07-16-2017 04:56 PM
Last Visited ‎07-17-2017 02:14 PM
Total Messages Posted 3