I need to move data between PROD and DEV clusters with same HDP version and HA.
Prod = 8 DNs
DEV = 4 DNs
I would like to use distcp. What firewall rules do I need to setup?
1. Allow port 50010 from each Prod DN to each Dev DN - 32 rules.
2. Allow port 8020 from for active Prod NN to active Dev NN -1 rule.
3. Execute discp on Prod cluster
1.Do I need to apply FW rules also in opposite way - from Dev to Prod?
2. Any idea to simplify this process - performance is not so important goal is to move data from hdfs to hdfs
Thanks in advance,
@Jozef Carada Check this out https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_administration/content/distcp_between_ha...