Hello, Has anyone tired using aws EFS mounts as yarn scratch and hdfs directories? The data long term will sit in S3. Please let me know your thoughts. Thanks,
... View more
Take a look at these: https://stackoverflow.com/questions/18477983/what-exactly-non-dfs-used-means https://community.hortonworks.com/questions/42122/hdfs-non-dfs-used.html
... View more
Hello, DFS used 2.3 PB will go up 2.6 PB Remaining 918.4 TB will go down to 918.1 non DFS should not change. let me know if you need more info.
... View more
What is your clusters replication factor? If your replication is 3X then 100 GB x 3 = 300 GB. 300 GB will be the about of hdfs storage used.
... View more
In my case the data will be moved to HDFS via an ingestion process. We are looking to know if the user queries can be run as is or if they will need to invest a lot of time in doing changes to there code.
... View more
