Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How many NameNodes needed on Backup Cluster

avatar
Explorer

The customer wants to setup a backup to a backup cluster. They want to know how many NameNodes are required on the Backup Cluster. They want to know if there is some rule of thumb. The constraints for the backup cluster is below

    • Constraints
      • Backup window: 1am-5am ET (12am-4am CT)
      • Want to have a yarn job that runs Falcon backup jobs in another queue when the cluster isn’t busy - that way it doesn't pile up for the night.
      • The clusters will be in Amazon, so they want the backup cluster to be backed by Amazon S3 storage (being used as HDFS)
    1 ACCEPTED SOLUTION

    avatar

    Hi @mhendricks you should use 2 Namenodes to increase the availability of the backup cluster. This is especially important if you only have limited window for your backup (like 1-5am).

    This might also be helpful:

    https://community.hortonworks.com/questions/4135/what-to-backup-and-how-only-metadata-not-data.html

    https://community.hortonworks.com/questions/394/what-are-best-practices-for-setting-up-backup-and.ht...

    View solution in original post

    4 REPLIES 4

    avatar
    Master Mentor

    @mhendricks

    2 namenode

    Active and passive setup

    I would treat backup cluster setup "not config" almost same as production

    Configs = CPU & Memory

    avatar
    Master Mentor
    @mhendricks

    The overall flows looks good.

    avatar

    Hi @mhendricks you should use 2 Namenodes to increase the availability of the backup cluster. This is especially important if you only have limited window for your backup (like 1-5am).

    This might also be helpful:

    https://community.hortonworks.com/questions/4135/what-to-backup-and-how-only-metadata-not-data.html

    https://community.hortonworks.com/questions/394/what-are-best-practices-for-setting-up-backup-and.ht...

    avatar
    Master Mentor

    +1 cost savings are great but not with backups