Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hadoop Ecosystem Log Monitoring and Analysis

Solved Go to solution
Highlighted

Hadoop Ecosystem Log Monitoring and Analysis

Rising Star

Hi,

I am about to setup an Hadoop ecosystem. I have some practices with the environment setup but not with monitoring/managing the logs with an external solution.

Since there will be a lot of applications to print logs and yet so much log files, I will need a solution which will be able to monitor them (Will be great if I can also manage and analyze them). I have heard about ELK and some others but I would like to hear from some experienced people.

Can you recommend one such solution?

Btw I'll use Ambari so it'd be good if you add its compatibility information whether or not can be managed by Ambari.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Hadoop Ecosystem Log Monitoring and Analysis

Mentor

In past couple of versions of Ambari, we introduced Log Search UI to help you do just that, you can view across all of the logs in the HDP stack from a single viewer. http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/ch_ambari_log_s...

It parses the logs, indexes them and you can do searches on log levels

View solution in original post

3 REPLIES 3
Highlighted

Re: Hadoop Ecosystem Log Monitoring and Analysis

Mentor

In past couple of versions of Ambari, we introduced Log Search UI to help you do just that, you can view across all of the logs in the HDP stack from a single viewer. http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/ch_ambari_log_s...

It parses the logs, indexes them and you can do searches on log levels

View solution in original post

Highlighted

Re: Hadoop Ecosystem Log Monitoring and Analysis

Rising Star

Hi @Artem Ervits,

Thank you for your answer.

Is that possible to integrate Log Search UI with Kibana?

Highlighted

Re: Hadoop Ecosystem Log Monitoring and Analysis

Mentor

@Sedat Kestepe probably can, we don't restrict you from taking the logs and do your own analysis on it, but why would you want to? Take a look at the rest of the documentation, there is a cool UI to slice and dice the logs. Here's a screenshot https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/viewing_servic...

please consider accepting the answer.

Don't have an account?
Coming from Hortonworks? Activate your account here