Support Questions
Find answers, ask questions, and share your expertise

can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

Expert Contributor

Hi Team,

I have 3 virtual machines in HDP cluster ,if i have huge capacity in data nodes disk in TBs so can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var .

I know then if my disk has some issue then all data will be affected

basically i wanted to know if my data node disks have lot of space in TBs, so do you recommend creating diff mounts on same data node disks for diff purposes like /usr,/var and storing NN SN JT data

Also each HDP version data is in /usr/hdp

1 ACCEPTED SOLUTION

Accepted Solutions

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

Super Guru

@ripunjay godhani

Don't mount all the partitions on same disk, it will create lot of disk I/O.

I would suggest to partition the disk according to your requirement and use dedicated disks for each component like DNs/NNs etc.

Also, Please have a look at below links for Hadoop performance tuning

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-1/

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-2/

View solution in original post

4 REPLIES 4

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

Super Guru

@ripunjay godhani

Is this a sandbox, just for testing and experimenting? If yes, then it's fine. For anything else, no. This is not recommended.

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

Super Guru

@ripunjay godhani

Don't mount all the partitions on same disk, it will create lot of disk I/O.

I would suggest to partition the disk according to your requirement and use dedicated disks for each component like DNs/NNs etc.

Also, Please have a look at below links for Hadoop performance tuning

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-1/

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-2/

View solution in original post

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

Expert Contributor

Thanks a lot Kuldeep i agree and thats why i wanted suggestions from experts like you 🙂

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

I think there is an HCC article on this very topic, but https://martin.atlassian.net/wiki/x/EoC3Ag is a blog post I wrote back in mid-2015 on this subject as well in case it helps any. Good luck!