Yep, what @Joe Widen said and sprinkle in a bit of "it depends" on top for good measure. If I put my consulting cap on again, I'd ask "what are you trying to do?". Meaning, is the job in question more about memory usage or overall compute time to complete. There will be factors at play that don't let you have a perfect yes or no answer to the question above and I'd recommend picking an indicative use case of your data processing and have a small bake-off on your cluster based on what is most important to you if you really need to a more data-driven decision. Chances are that your decision will be based on what you plan on developing in going forward more so than the answer to this question about memory usage. ... View more

@Darpan Patel flume is one directional, you can't push data out of hdfs to fs. We have Apache NiFi that you can leverage for moving logs to and from hdfs. ... View more

Thanks @Artem Ervits ... View more

Also I had meet the second problem? How can I update the connect timeout setting? There were test environment not enough resources so I don't mind it. ... View more

@Darpan Patel Perfect! Please accept one of the answers to close the thread as part of the best practice. ... View more

@Darpan Patel Darpan, I have one question related to what you did. I am newbie to Kerberos. I am actually running a similar configuration, where I have AD that holds all principals. Regarding what you have said: >>After Kerberization I created a user "ambari­-user/ambari-Host_name_here@KDCRealm.com you did this in the AD right ? >>And also created a key tab, copied on the ambari -server machine How did you do that? You created the keytab at the ambari-server host ? or created it in AD and somehow you copied the keytab to /etc/security/keytabs of your ambari server host ? ... View more

@Darpan Patel Ok. If issue is resolved then please accept one of the answers to close the thread. ... View more