Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hadoop Ecosystem Log Monitoring and Analysis

avatar
Expert Contributor

Hi,

I am about to setup an Hadoop ecosystem. I have some practices with the environment setup but not with monitoring/managing the logs with an external solution.

Since there will be a lot of applications to print logs and yet so much log files, I will need a solution which will be able to monitor them (Will be great if I can also manage and analyze them). I have heard about ELK and some others but I would like to hear from some experienced people.

Can you recommend one such solution?

Btw I'll use Ambari so it'd be good if you add its compatibility information whether or not can be managed by Ambari.

1 ACCEPTED SOLUTION

avatar
Master Mentor

In past couple of versions of Ambari, we introduced Log Search UI to help you do just that, you can view across all of the logs in the HDP stack from a single viewer. http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/ch_ambari_log_s...

It parses the logs, indexes them and you can do searches on log levels

View solution in original post

3 REPLIES 3

avatar
Master Mentor

In past couple of versions of Ambari, we introduced Log Search UI to help you do just that, you can view across all of the logs in the HDP stack from a single viewer. http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/ch_ambari_log_s...

It parses the logs, indexes them and you can do searches on log levels

avatar
Expert Contributor

Hi @Artem Ervits,

Thank you for your answer.

Is that possible to integrate Log Search UI with Kibana?

avatar
Master Mentor

@Sedat Kestepe probably can, we don't restrict you from taking the logs and do your own analysis on it, but why would you want to? Take a look at the rest of the documentation, there is a cool UI to slice and dice the logs. Here's a screenshot https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/viewing_servic...

please consider accepting the answer.