Users can upload and create directories using file view without home directory on /users in HDP 2.4 Sandbox?

[#1]

Guys,

I was trying Files View which comes configured with Ambari in HDP Sandbox 2.4. To my surprise, I could upload the files and create directories under root and other directories through Ambari Files View even though I did not create the /user/<userName> directory on the HDFS. This is true for all the newly added users through Ambari UI and also for the users admin/maria_dev,etc.

Is it a bug or limitation removal for HDP 2.4 Sandbox ?

Regards,

SS!

[#2]

Have a look in Ambari >> Services >> HDFS >> Configs >> Advanced ...

In the "Filter" box top right, type the text "proxy". In custom core-site, you will probably find the following settings:

hadoop.proxyuser.root.groups=*
hadoop.proxyuser.root.hosts=*

which essentially gives Linux "root" privileges to the Ambari service. See this for more information. In a production system, you would probably want to use the syntax

hadoop.proxyuser.ambariusr.groups=* 

instead, where ambariusr is not root, and has limited privileges.

More information on Hadoop Proxy Users can be found here (Apache Hadoop site).

Hope this helps.

[#3]

I was interested to see if this was really having to do with the HDFS Ambari View or just with the HDFS permissions themselves, so I followed some notes in https://github.com/HortonworksUniversity/Essentials/blob/master/demos/mapreduce/README.md to create a local user "it1" as well as a home directory for it on HDFS. Then I found out I could do the same wide-open writes as you described. NOTE: I removed a bunch of the irrelevant folder contents to shorten the listing.

[root@sandbox ~]# su - it1

[it1@sandbox ~]$ hdfs dfs -ls /
Found 13 items
  ... DELETED LINES ...
drwxr-xr-x   - hdfs   hdfs            0 2016-03-31 17:17 /user
[it1@sandbox ~]$ hdfs dfs -put /etc/group /groups.txt
[it1@sandbox ~]$ hdfs dfs -ls /
Found 14 items
  ... DELETED LINES ...
-rw-r--r--   3 it1    hdfs         1196 2016-05-16 18:21 /groups.txt
drwxr-xr-x   - hdfs   hdfs            0 2016-03-31 17:17 /user

[it1@sandbox ~]$ hdfs dfs -put /etc/group /user/groups.txt
[it1@sandbox ~]$ hdfs dfs -ls /user
Found 14 items
  ... DELETED LINES ...
-rw-r--r--   3 it1       hdfs       1196 2016-05-16 18:22 /user/groups.txt
drwxr-xr-x   - it1       hdfs          0 2016-05-16 18:21 /user/it1
drwxr-xr-x   - maria_dev hdfs          0 2016-05-09 19:33 /user/maria_dev

Since I was able to put these files into / and /user even though I should not have been able to, it shows it has nothing to do with the Ambari View. My next hunch was if the "hadoop superuser" nuclear option, described in https://martin.atlassian.net/wiki/x/AoCpAQ, was used, but the dfs.permissions.superusergroup was still set to hdfs and I verified that root, maria_dev, and it1 were not in that group with a "lid -g hdfs" command.

Then it dawned on me that like I found out when playing with Ranger in my https://github.com/HortonworksUniversity/Essentials/blob/master/demos/ranger/README.md demo (which had a wide-open policy in Ranger for Hive) that there might be a similar general rule established in Ranger for HDFS and as the follow screenshot shows, there is.

Easy fix, just disable it and save it (and then wait at least 30 seconds for the cache rule to expire) and try it again.

[it1@sandbox ~]$ hdfs dfs -put /etc/group /groups2.txt
put: Permission denied: user=it1, access=WRITE, inode="/groups2.txt._COPYING_":hdfs:hdfs:drwxr-xr-x

[it1@sandbox ~]$ hdfs dfs -put /etc/group /user/groups2.txt

put: Permission denied: user=it1, access=WRITE, inode="/user/groups2.txt._COPYING_":hdfs:hdfs:drwxr-xr-x

Yep, that's it! Now... for the Sandbox, I'd just re-enable that rule as we're just messin 'round with this environment anyway!! Good luck!!