Yes Harsh, it's number of blocks. Block count is 6 Million. Deleted unwanted small files, now the cluster health is good   Is there any limit that a datanode should have only x no. of blocks? ... View more

Hi, I don't think that's possible given that most applications are based on HDFS semantics (strong consistency, POSIX compatible), and S3 simply isn't designed as a file system (eventual consistency, blob store). Plus, you lose data locality.   As far as I know, most cloud use cases still use HDFS as temporary, intermediate storage, and use S3 as permanent, eventual storage.   There've been several studies in using HDFS as meta store, and cloud as data store, but that's a huge work (see HDFS-9806) and probably in the Hadoop 4/CDH 7 timeframe.   Hope this helps. ... View more