Data replication in different cluster

[#1]

Hi,

We are planning to create a new cluster intended for different set of users. Currently we have data coming into our existing cluster in multiple ways. Mainly through Kafka and other batch jobs pushing data in regular intervals.

Also, data can be modified after replicated to new cluster. In this case, we want copy only delta instead of copy everything.

What would be an ideal tool for this job? Thanks in Advance!

[#2]

use DistCp (Distributed Copy). this is the distcp command.

OR

you can use Apache Nifi if you want a GUI driven solution