Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

avatar
Expert Contributor

Hi Team,

I have 3 virtual machines in HDP cluster ,if i have huge capacity in data nodes disk in TBs so can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var .

I know then if my disk has some issue then all data will be affected

basically i wanted to know if my data node disks have lot of space in TBs, so do you recommend creating diff mounts on same data node disks for diff purposes like /usr,/var and storing NN SN JT data

Also each HDP version data is in /usr/hdp

1 ACCEPTED SOLUTION

avatar
Master Guru

@ripunjay godhani

Don't mount all the partitions on same disk, it will create lot of disk I/O.

I would suggest to partition the disk according to your requirement and use dedicated disks for each component like DNs/NNs etc.

Also, Please have a look at below links for Hadoop performance tuning

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-1/

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-2/

View solution in original post

4 REPLIES 4

avatar
Super Guru

@ripunjay godhani

Is this a sandbox, just for testing and experimenting? If yes, then it's fine. For anything else, no. This is not recommended.

avatar
Master Guru

@ripunjay godhani

Don't mount all the partitions on same disk, it will create lot of disk I/O.

I would suggest to partition the disk according to your requirement and use dedicated disks for each component like DNs/NNs etc.

Also, Please have a look at below links for Hadoop performance tuning

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-1/

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-2/

avatar
Expert Contributor

Thanks a lot Kuldeep i agree and thats why i wanted suggestions from experts like you 🙂

avatar

I think there is an HCC article on this very topic, but https://martin.atlassian.net/wiki/x/EoC3Ag is a blog post I wrote back in mid-2015 on this subject as well in case it helps any. Good luck!