question Re: Shared Storage for HDFS HA in Archives of Support Questions (Read Only)

Shared Storage for HDFS HA

Nishan — Fri, 16 Sep 2022 08:56:02 GMT

Hi All,

I am trying to configure HA hdfs using shared directory and planning to use NAS.I understand that it just need to be mounted on my name node machines.Is there ball park size that I could look at.These are non production servers that I am playing with,so there will not be a huge load and the cluster is very small.

Thanks,

Nishan

Re: Shared Storage for HDFS HA

Darren — Wed, 26 Mar 2014 23:00:14 GMT

Hi Nishan,

You should not use shared directory (NFS HA). Use Quorum Journal for HDFS High Availability. It's easier to set up and offers true HA.

Thanks,

Darren

Re: Shared Storage for HDFS HA

Nishan — Thu, 27 Mar 2014 15:24:47 GMT

Thanks Darren.I understand qourum provides true HA.My cluster has 4 machines and I want to run journal node on all.The manager GUI prevents from confuguring HA in this case as it needs an odd number of journal nodes running.I know its more beneficial to go with an odd number so that we can have the system running even if some machines are down.But this is kind of a test scenario.

1.Do we have a way to configure HA for qorum with an even number for journal nodes.If there is then I would defintely like to go that route.

Thanks,

Nishanth

Re: Shared Storage for HDFS HA

Harsh J — Fri, 28 Mar 2014 05:18:30 GMT

The even number disallowance has nothing to do with availability.

An odd number of nodes is required to form a "quorum". You cannot have definite majority to a vote request with even number of nodes in a quorum. In an odd set, there's always a majority to a vote's result, and thereby the requirement.

Hope this helps you make sense on why CM is requiring you to configure your QJM properly, with an odd number of nodes.

Re: Shared Storage for HDFS HA

Nishan — Fri, 28 Mar 2014 21:24:19 GMT

Thanks Harsh.Got your point.