How many NameNodes needed on Backup Cluster

[#1]

The customer wants to setup a backup to a backup cluster. They want to know how many NameNodes are required on the Backup Cluster. They want to know if there is some rule of thumb. The constraints for the backup cluster is below

• Constraints
  • Backup window: 1am-5am ET (12am-4am CT)
  • Want to have a yarn job that runs Falcon backup jobs in another queue when the cluster isn’t busy - that way it doesn't pile up for the night.
  • The clusters will be in Amazon, so they want the backup cluster to be backed by Amazon S3 storage (being used as HDFS)

[#2]

@mhendricks

2 namenode

Active and passive setup

I would treat backup cluster setup "not config" almost same as production

Configs = CPU & Memory

[#3]

@mhendricks

The overall flows looks good.

[#4 · Accepted solution]

Hi @mhendricks you should use 2 Namenodes to increase the availability of the backup cluster. This is especially important if you only have limited window for your backup (like 1-5am).

This might also be helpful:

https://community.hortonworks.com/questions/4135/what-to-backup-and-how-only-metadata-not-data.html

https://community.hortonworks.com/questions/394/what-are-best-practices-for-setting-up-backup-and.ht...

[#5]

+1 cost savings are great but not with backups