Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Change NameNode and Datanode directories to Azure Data Lake Storage

Change NameNode and Datanode directories to Azure Data Lake Storage

Explorer

I just deployed my cluster on Azure using Cloudbreak. Right now my name node and datanode directories are: /hadoopfs/fs1/hdfs/namenode and /hadoop/hdfs/data,/hadoopfs/fs1/hadoop/hdfs/data,/mnt/resource/hadoop/hdfs/data respectively. I want to change this to a location on my Data Lake Store.
When I just paste the path starting with "adl:/" it throws an error. I setup the access and everything. I just want to change the directories to save space on my local FS.
Also, I am getting alerts like:

Capacity Used:[100%, 28672], Capacity Remaining:[0].

I have 110 GB RAM on my master node. Why is it already used up?

2 REPLIES 2
Highlighted

Re: Change NameNode and Datanode directories to Azure Data Lake Storage

@kskp

Have you set fs.defaultFS to ADLS as well? If so, note that it is not an official setup supported by Hortonworks, and it might cause HDFS calculating free space incorrectly.

Hope this helps!

Highlighted

Re: Change NameNode and Datanode directories to Azure Data Lake Storage

Explorer

I did. I just had to change the value of Reserved Space for HDFS in HDFS configs. Thanks for commenting.

Don't have an account?
Coming from Hortonworks? Activate your account here