Archives of Support Questions (Read Only)

melvinmendoza · ‎04-05-2017

Additional Questions:

Do we need to set Raid 1 in our SSD ?
Does our configuration in hadoop okay with Raid 1/5?

Master 1 (NameNode)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1(Caching)
3 x 2TB Raid 5 (hadoop)

Worker 2 (DataNode1)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1 (Caching)
3 x 2TB Raid 5 (hadoop)

Worker 3 (DataNode2)

Raid Configuration:

2 x 300GB – Raid 1 (OS)
2 x 400GB SSD Raid 1(Caching)
3 x 2TB Raid 5(hadoop)

pminovic · ‎04-07-2017

For NN data some fault-tolerant RAID, like 1, 5, or 10 is fine. On worker nodes, for hdfs data you should use JBOD or RAID-0 per disk (so that you have 3 mount points). RAID-1 for OS on all nodes is fine. I'm not sure what do you mean by "cashing".

View solution in original post

melvinmendoza · ‎04-07-2017

Great! will take note of that, how about in Spark, should I set a buffer on SSDs for DISK persist?

pminovic · ‎04-07-2017

Not sure about Spark, but IMO you can do that when you configure HDFS, put SSD nodes in another Ambari config group and set the space not to be used by HDFS.

Cloudera Community

Archives of Support Questions (Read Only)

Do we config our hadoop right? JBOD vs RAID