question Re: Do we config our hadoop right? JBOD vs RAID in Support Questions

Do we config our hadoop right? JBOD vs RAID

melvinmendoza — Wed, 05 Apr 2017 08:08:10 GMT

Additional Questions:

Do we need to set Raid 1 in our SSD ?
Does our configuration in hadoop okay with Raid 1/5?

Master 1 (NameNode)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1(Caching)
3 x 2TB Raid 5 (hadoop)

Worker 2 (DataNode1)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1 (Caching)
3 x 2TB Raid 5 (hadoop)

Worker 3 (DataNode2)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1(Caching)
3 x 2TB Raid 5(hadoop)

Re: Do we config our hadoop right? JBOD vs RAID

namaheshwari — Wed, 05 Apr 2017 10:52:46 GMT

Ping @Arpit Agarwal

Re: Do we config our hadoop right? JBOD vs RAID

tn — Thu, 06 Apr 2017 06:32:05 GMT

http://blog.cloudera.com/blog/2015/01/how-to-deploy-apache-hadoop-clusters-like-a-boss/

Re: Do we config our hadoop right? JBOD vs RAID

ArpitAgarwal — Thu, 06 Apr 2017 23:33:37 GMT

Using RAID can reduce the availability and fault tolerance of HDFS. It certainly reduces the overall performance as compared to JBOD.

We strongly recommend configuring your disks as JBOD since HDFS already stores data redundantly by replicating across nodes/racks and can automatically recover from disk and node failures.

Re: Do we config our hadoop right? JBOD vs RAID

melvinmendoza — Fri, 07 Apr 2017 07:52:13 GMT

In my configuration, setting RAID 1 on OS and in Caching is okay? or should I change it to RAID 0?

Re: Do we config our hadoop right? JBOD vs RAID

pminovic — Fri, 07 Apr 2017 08:00:34 GMT

For NN data some fault-tolerant RAID, like 1, 5, or 10 is fine. On worker nodes, for hdfs data you should use JBOD or RAID-0 per disk (so that you have 3 mount points). RAID-1 for OS on all nodes is fine. I'm not sure what do you mean by "cashing".

Re: Do we config our hadoop right? JBOD vs RAID

melvinmendoza — Fri, 07 Apr 2017 11:46:48 GMT

@Predrag Minovic meaning my config should look like this:

Master 1 (NameNode)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1(Caching) - HOT (for HDFS)
3 x 2TB Raid 5 (hadoop) - one mount point WARM,COLD,ARCHIVE

Worker 2 (DataNode1)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1 (Caching) HOT (for HDFS)
3 x 2TB Raid 0 (hadoop) - 3 mount points WARM,COLD,ARCHIVE

Worker 3 (DataNode2)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1 (Caching) HOT (for HDFS)
3 x 2TB Raid 0 (hadoop) - 3 mount points WARM,COLD,ARCHIVE

Should we also remove RAID 1 to SSD?

Re: Do we config our hadoop right? JBOD vs RAID

pminovic — Fri, 07 Apr 2017 12:41:11 GMT

Okay, now I understand what do you mean by "cashing". Yes, you can remove RAID-1 on SSD's, then you can experiment with One_SSD and All_SSD policies, either way there are multiple replicas, so no need for RAID. And by the way there is no storage policy for NN, if possible it will be good to move 2x400G SSD from NN to worker nodes.

Re: Do we config our hadoop right? JBOD vs RAID

melvinmendoza — Fri, 07 Apr 2017 13:00:49 GMT

Great! will take note of that, how about in Spark, should I set a buffer on SSDs for DISK persist?

Re: Do we config our hadoop right? JBOD vs RAID

melvinmendoza — Fri, 07 Apr 2017 13:53:45 GMT

Thanks! @Tom N

Re: Do we config our hadoop right? JBOD vs RAID

melvinmendoza — Fri, 07 Apr 2017 13:53:55 GMT

Thanks ! @Arpit Agarwal

Re: Do we config our hadoop right? JBOD vs RAID

pminovic — Fri, 07 Apr 2017 14:16:25 GMT

Not sure about Spark, but IMO you can do that when you configure HDFS, put SSD nodes in another Ambari config group and set the space not to be used by HDFS.