@Mondi    It is not compulsory to enable SSL but recommended to prevent the passage of plain text key material between the KMS and HDFS data nodes. You can continue to install Java Keystore KMS without adding SSL configurations.   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

@Mondi As far as I know real time replication is not possible but tow cluster can be in sync using BDR feature. https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_bdr_replication_intro.html ... View more

Hello @Mondi ,   thank you for raising your question about why application logs being deleted from the nodes after the applications finished running and why is it happening, how to keep them in place.   When log aggregation is enabled with the 'yarn.log-aggregation-enable = true' [1] you will observe the behaviour described: after the logs are aggregated to HDFS, the logs are immediately deleted from the local file system. Log aggregation does not start until the application is finished. If you need to keep the logs and some other temporary files on the local node for troubleshooting and you have log aggregation turned on then you can use yarn.nodemanager.delete.debug-delay-sec . This is set to 0 seconds by default causing the immediate delete.   Should you disable log aggregation, non aggregated logs are kept for yarn.nodemanager.log.retain-seconds = 10800 seconds (3*3600 seconds or 3 hours). After that the NodeManager will delete the log files.   Please let us know if your enquiries been addressed!   Thank you: Ferenc   [1] https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml ... View more

@Mondi    You would still need to secure your cluster since any user can be impersonated in a non kerberised cluster. Refer https://blog.cloudera.com/how-to-secure-internet-exposed-apache-hadoop/ for more details on securing your cluster. ... View more

Hello, This topic is described at length in the official documentation - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_ports.html  Kind regards, Julius ... View more

@Mondi    It is highly recommended to install kerberos as a prerequisite before using sentry. HiveServer2 and the Hive Metastore (HMS) running with strong authentication. For HiveServer2, strong authentication is either Kerberos or LDAP. For the Hive Metastore, only Kerberos is considered strong authentication (to override, see Securing the Hive Metastore). Below document mentions the prerequisites in detail: https://docs.cloudera.com/documentation/enterprise/latest/topics/sg_sentry_before_you_install.html#prerequisites   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more