@Prateek Behera The disk quota in your case works fine as expected. HDFS by default has a replication factor of 3 as you can see in the 3rd column of your CLI output. 500MB *3 (replication factor) = 1.5 GB > 1GB (quota). ... View more

@szetszwo, you can do this with haodop impersonation by setting the env variable $HADOOP_PROXY_USER=foo ... View more

hdfs debug recoverLease command should be used to close the file. The complete syntax is like below: hdfs debug recoverLease -path <path-of-the-file>[-retries <retry-times>] ... View more

Apache Hadoop has an official wiki page that summarized major changes across 1.x, 2.x. Hope that helps. https://wiki.apache.org/hadoop/Roadmap ... View more

You will need to add auth_to_local rule in core-site.xml to map dgiri@mytestdomain.com to dgiri RULE:[2:$1](.*@mytestdomain.com)s/@.*// More defailt of how to write auth_to_local rules can be found here: https://hortonworks.com/blog/fine-tune-your-apache-hadoop-security-settings/ ... View more

bq. From what I understood every files use a minimum of 1 block in HDFS. No. It is not true. You can have file of size 0, they are likely to be created on NN but no block being allocated and/or streamed to DNs yet. Check your files and you will have 4 files (8694-8690=4) of size 0. ... View more

decrypt operation needs POST instead of GET. Check the samples form KMS doc below. Decrypt Encrypted Key REQUEST: POST http://HOST:PORT/kms/v1/keyversion/<version-name>/_eek?ee_op=decrypt Content-Type: application/json { "name" : "<key-name>", "iv" : "<iv>", //base64 "material" : "<material>", //base64 } RESPONSE: 200 OK Content-Type: application/json { "name" : "EK", "material" : "<material>", //base64 } ... View more

Please check hadoop wiki at https://wiki.apache.org/hadoop though some of the info maybe outdated. ... View more

@Xiong Duan - fully reload configuration files like hdfs-site.xml/core-site.xml without a restart is not supported. However, you can change certain configuration keys without a restart using hdfs dfsadmin [-reconfig <namenode|datanode> <host:ipc_port> <start|status|properties>] ... View more