Support Questions
Find answers, ask questions, and share your expertise

How many NameNodes needed on Backup Cluster

Solved Go to solution
Highlighted

How many NameNodes needed on Backup Cluster

New Contributor

The customer wants to setup a backup to a backup cluster. They want to know how many NameNodes are required on the Backup Cluster. They want to know if there is some rule of thumb. The constraints for the backup cluster is below

    • Constraints
      • Backup window: 1am-5am ET (12am-4am CT)
      • Want to have a yarn job that runs Falcon backup jobs in another queue when the cluster isn’t busy - that way it doesn't pile up for the night.
      • The clusters will be in Amazon, so they want the backup cluster to be backed by Amazon S3 storage (being used as HDFS)
    1 ACCEPTED SOLUTION

    Accepted Solutions
    Highlighted

    Re: How many NameNodes needed on Backup Cluster

    Hi @mhendricks you should use 2 Namenodes to increase the availability of the backup cluster. This is especially important if you only have limited window for your backup (like 1-5am).

    This might also be helpful:

    https://community.hortonworks.com/questions/4135/what-to-backup-and-how-only-metadata-not-data.html

    https://community.hortonworks.com/questions/394/what-are-best-practices-for-setting-up-backup-and.ht...

    View solution in original post

    4 REPLIES 4
    Highlighted

    Re: How many NameNodes needed on Backup Cluster

    @mhendricks

    2 namenode

    Active and passive setup

    I would treat backup cluster setup "not config" almost same as production

    Configs = CPU & Memory

    Highlighted

    Re: How many NameNodes needed on Backup Cluster

    @mhendricks

    The overall flows looks good.

    Highlighted

    Re: How many NameNodes needed on Backup Cluster

    Hi @mhendricks you should use 2 Namenodes to increase the availability of the backup cluster. This is especially important if you only have limited window for your backup (like 1-5am).

    This might also be helpful:

    https://community.hortonworks.com/questions/4135/what-to-backup-and-how-only-metadata-not-data.html

    https://community.hortonworks.com/questions/394/what-are-best-practices-for-setting-up-backup-and.ht...

    View solution in original post

    Highlighted

    Re: How many NameNodes needed on Backup Cluster

    Mentor

    +1 cost savings are great but not with backups