Support Questions
Find answers, ask questions, and share your expertise

Change NameNode and Datanode directories to Azure Data Lake Storage

Explorer

I just deployed my cluster on Azure using Cloudbreak. Right now my name node and datanode directories are: /hadoopfs/fs1/hdfs/namenode and /hadoop/hdfs/data,/hadoopfs/fs1/hadoop/hdfs/data,/mnt/resource/hadoop/hdfs/data respectively. I want to change this to a location on my Data Lake Store.
When I just paste the path starting with "adl:/" it throws an error. I setup the access and everything. I just want to change the directories to save space on my local FS.
Also, I am getting alerts like:

Capacity Used:[100%, 28672], Capacity Remaining:[0].

I have 110 GB RAM on my master node. Why is it already used up?

2 REPLIES 2

@kskp

Have you set fs.defaultFS to ADLS as well? If so, note that it is not an official setup supported by Hortonworks, and it might cause HDFS calculating free space incorrectly.

Hope this helps!

Explorer

I did. I just had to change the value of Reserved Space for HDFS in HDFS configs. Thanks for commenting.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.