I have a HDP Cluster whose 60% space is consumed. So, when I am running hive queries from Zeppelin , the space consumed on hdfs increases to 89% resulting in continues warning from Ambari. I found that the space in "/tmp/hive/hive" gradually increases . I had to kill the yarn job to stop the query . On killing the extra HDFS space consumed is released, reverting back to 60% hdfs used. As per my understanding "/tmp/hive/hive" directory is being used to store the temp data while executing MapReduce. So my question is how much storage be left on HDFS , so that my Cluster does not gets exhausted while running queries? Please correct me on the below statement :- (Total HDFS Size = Size of Data Stored + Buffered Storage to execute queries) To add more details for the above question :- Replication Factor =3 Data stored Format = Parquet ... View more

In our application, we are having only LDAP (Single Server) for external authentication. Why do we need a sync of users in Ambari internal database. Can this authentication be done directly on LDAP , without having a copy of user\s in internal database. Example :- While applying security on Zeppelin through Shiro , we just give the details of LDAP server , and it does the authentication. Similarity is there any way we can directly authenticate from LDAP . ... View more