@Mondi As far as I know real time replication is not possible but tow cluster can be in sync using BDR feature. https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_bdr_replication_intro.html ... View more

Hello @Mondi ,   thank you for raising your question about why application logs being deleted from the nodes after the applications finished running and why is it happening, how to keep them in place.   When log aggregation is enabled with the 'yarn.log-aggregation-enable = true' [1] you will observe the behaviour described: after the logs are aggregated to HDFS, the logs are immediately deleted from the local file system. Log aggregation does not start until the application is finished. If you need to keep the logs and some other temporary files on the local node for troubleshooting and you have log aggregation turned on then you can use yarn.nodemanager.delete.debug-delay-sec . This is set to 0 seconds by default causing the immediate delete.   Should you disable log aggregation, non aggregated logs are kept for yarn.nodemanager.log.retain-seconds = 10800 seconds (3*3600 seconds or 3 hours). After that the NodeManager will delete the log files.   Please let us know if your enquiries been addressed!   Thank you: Ferenc   [1] https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml ... View more

@Mondi    You would still need to secure your cluster since any user can be impersonated in a non kerberised cluster. Refer https://blog.cloudera.com/how-to-secure-internet-exposed-apache-hadoop/ for more details on securing your cluster. ... View more

Hello, This topic is described at length in the official documentation - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_ports.html  Kind regards, Julius ... View more