what commodity or appliance hardware ? and how can i know that what my cluster is ?

[#1]

what commodity or appliance hardware ? and how can i know that what my cluster is ?

I am using HDP 2.1.2 with support of TERADATA

i am using DEV and PROD clusters(130TB)

Prod cluster has 4 data node(24 cores +125gb ram)2 name nodes(32 cores +300gb ram) 1 Edge node with configure

what are the other hardware configuration need to verify?

[#2 · Accepted solution]

@sankar rao

not sure if I understand your question. 4 data nodes, 2 masters and 1 edge looks fine. Namenode seem to have a lot of cores and memory. Data nodes look fine. What else is running on those name nodes that require 32 cores and 300 GB RAM? You didn't mention edge node configuration but that can be very flexible. normally I recommend 3 masters - 2 namenodes plus redundancy for everything including Hive metastore, Hiver Server 2, 3 zookeepers (with their own dedicate disks on each node), 3 quorum journal for namenode shared edits and fail over (2x500 GB SAS disks in RAID 1). So, ideally you want three masters.

Is that what your question is?

[#3]

@mqureshi

Thanks for your time..I am new to this hadoop env and i just want to know that i am using commodity or appliance hardware? how can i call my hardware? can i call it as commodity or appliance hardware?

what is commodity hardware and appliance hardware? what is different b/w them?

[#4]

oh. If it's delivered by Teradata to you, then it's an appliance. If you bought your own hardware from let's say a vendor like HP, Dell or IBM then it's commodity - although term commodity hardware is not used anymore because people confuse it with personal laptops. A better term is "industry standard" hardware.

If you purchase a turnkey solution where everything came installed from the vendor like Teradata, then it's an appliance. If you bought your own hardware and then did an install and whole setup yourself, then it's industry standard hardware (aka commodity hardware).

[#5]

@mqureshi

NICI Nice..i got the answer thanks you so much...