question can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var in Archives of Support Questions (Read Only)

can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

ripu — Mon, 01 Aug 2016 13:11:42 GMT

Hi Team,

I have 3 virtual machines in HDP cluster ,if i have huge capacity in data nodes disk in TBs so can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var .

I know then if my disk has some issue then all data will be affected

basically i wanted to know if my data node disks have lot of space in TBs, so do you recommend creating diff mounts on same data node disks for diff purposes like /usr,/var and storing NN SN JT data

Also each HDP version data is in /usr/hdp

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

mqureshi — Mon, 01 Aug 2016 13:26:27 GMT

@ripunjay godhani

Is this a sandbox, just for testing and experimenting? If yes, then it's fine. For anything else, no. This is not recommended.

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

KuldeepK — Mon, 01 Aug 2016 13:32:27 GMT

@ripunjay godhani

Don't mount all the partitions on same disk, it will create lot of disk I/O.

I would suggest to partition the disk according to your requirement and use dedicated disks for each component like DNs/NNs etc.

Also, Please have a look at below links for Hadoop performance tuning

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-1/

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-2/

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

ripu — Mon, 01 Aug 2016 14:09:15 GMT

Thanks a lot Kuldeep i agree and thats why i wanted suggestions from experts like you 🙂

Re: can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var

LesterMartin — Mon, 08 Aug 2016 09:07:51 GMT

I think there is an HCC article on this very topic, but https://martin.atlassian.net/wiki/x/EoC3Ag is a blog post I wrote back in mid-2015 on this subject as well in case it helps any. Good luck!