Thank @Artem Ervits some of these links are helpful, however let me clarify a little more... As part of hadoop applications we will be generating logs at different levels by different components . You will typically have end user computing along with hadoop processes like hive pig MR etc. So what is the best practice there, given that the application may have some components outside the hadoop cluster, how can you get end to end view of what happened from logging for an application ... View more