@Neeraj Sabharwal AFAIK, there's no quotas management view in Ambari, I wrote a short tutorial on quotas, ### QUOTAS ###
# quotas can be number of files or size per directory, independent of each other
sudo -u hdfs hdfs dfs -mkdir /quotasdir
# requires superuser privileges
# set space quota of 1kb on a directory, can be k, m, g, etc.
sudo -u hdfs hdfs dfsadmin -setSpaceQuota 1k /quotasdir
# add a file
sudo -u hdfs hdfs dfs -touchz /quotasdir/1
# notice file is 0 bytes
sudo -u hdfs hdfs dfs -ls /quotasdir/
# for demo purposes, we need to upload a large file, larger than 1kb into directory, watch the prompt
sudo -u hdfs hdfs dfs -chown -R root:hdfs /quotasdir
hdfs dfs -put /root/install.log /quotasdir/
15/11/25 15:10:47 WARN hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /quotasdir is exceeded: quota = 1024 B = 1 KB but diskspace consumed = 402653184 B = 384 MB
at org.apache.hadoop.hdfs.server.namenode.DirectoryWithQuotaFeature.verifyStoragespaceQuota(DirectoryWithQuotaFeature.java:211)
at org.apache.hadoop.hdfs.server.namenode.DirectoryWithQuotaFeature.verifyQuota(DirectoryWithQuotaFeature.java:239)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.verifyQuota(FSDirectory.java:907)
# remove space quota
sudo -u hdfs hdfs dfsadmin -clrSpaceQuota /quotasdir
# try uploading again
hdfs dfs -put /root/install.log /quotasdir/
# ls on the directory
sudo -u hdfs hdfs dfs -ls /quotasdir/
# set name quota on directory, specifies how many files or directories per quota directory where # 2 is number of files or directories allowed in the quota dir
sudo -u hdfs hdfs dfsadmin -setQuota 3 /quotasdir
# add file, nothing happens
sudo -u hdfs hdfs dfs -touchz /quotasdir/1
# add another file and watch the prompt
sudo -u hdfs hdfs dfs -touchz /quotasdir/2
touchz: The NameSpace quota (directories and files) of directory /quotasdir is exceeded: quota=2 file count=3
# remove name quota
sudo -u hdfs hdfs dfsadmin -clrQuota /quotasdir
# now adding another file works
sudo -u hdfs hdfs dfs -touchz /quotasdir/2
# ls on the directory
sudo -u hdfs hdfs dfs -ls /quotasdir/
# An an extension to the count command of the HDFS shell reports quota values and the current count of names and bytes in use.
fs -count -q <directory>...<directory>With the -q option, also report the name quota value set for each directory, the available name quota remaining, the space quota value set, and the available space quota remaining. If the directory does not have a quota set, the reported values are none and inf.# make sure all quotas are cleared sudo -u hdfs hdfs dfsadmin -clrSpaceQuota /quotasdir
sudo -u hdfs hdfs dfsadmin -clrQuota /quotasdir
sudo -u hdfs hdfs dfs -count -q /quotasdir
none inf none inf 1 4 9447 /quotasdir
# set quotas and check again
sudo -u hdfs hdfs dfsadmin -setSpaceQuota 99g /quotasdir
sudo -u hdfs hdfs dfsadmin -setQuota 8888 /quotasdir
8888 8883 106300440576 106300412235 1 4 9447 /quotasdir
sudo -u hdfs hdfs dfs -count -q /quotasdir
# 8888 is number of files
# 106300440576 is 99GB
... View more
