Shared Storage for HDFS HA

[#1]

Hi All,

I am  trying to configure HA hdfs  using shared directory and planning to use NAS.I understand that it just need to be mounted on my name node machines.Is there  ball park size that I could look at.These are non production servers that I am playing with,so there will not be a huge load and the cluster is very small.

Thanks,

Nishan

[#2]

Hi Nishan,

You should not use shared directory (NFS HA). Use Quorum Journal for HDFS High Availability. It's easier to set up and offers true HA.

Thanks,

Darren

[#3]

Thanks Darren.I understand   qourum  provides  true HA.My cluster has 4 machines and I want to run journal node on all.The manager GUI prevents from  confuguring HA in this case as it  needs an odd number of journal nodes running.I know its more beneficial to go with an odd  number so that we can have the system running even if some machines are down.But this is  kind of a test scenario.

1.Do we have a  way to configure HA for qorum with an even number for journal nodes.If there is then I would defintely like to go that route.

Thanks,

Nishanth

[#4 · Accepted solution]

The even number disallowance has nothing to do with availability.

An odd number of nodes is required to form a "quorum". You cannot have definite majority to a vote request with even number of nodes in a quorum. In an odd set, there's always a majority to a vote's result, and thereby the requirement.

Hope this helps you make sense on why CM is requiring you to configure your QJM properly, with an odd number of nodes.

[#5]

Thanks Harsh.Got your point.