Reply
Highlighted
New Contributor
Posts: 8
Registered: ‎11-15-2013
Accepted Solution

How to enable TRACE logging for morphlines

Hi, 

 

I'm getting this log statement in my logs: 

WARN org.apache.flume.sink.solr.morphline.MorphlineHandlerImpl: Morphline morphlines.conf@morphline1 failed to process record:

 

In this article (http://blog.cloudera.com/blog/2013/07/morphlines-the-easy-way-to-build-and-integrate-etl-apps-for-ap... is suggested to increase the log level for morphlines to TRACE so you can see what happens to the message as it moves through the commands.

 

I didn't find an option to do this in the admin console, and when I was looking for log4j.properties I found one in my /etc/flume-ng/conf dir, but that only contains a configuration for flume.log and it looks like it isn't used.

I am getting a log file with the following name: flume-cmf-flume1-AGENT-localhost.localdomain.log, but I can't find the log4j properties that configures the logging for this.

 

Any idea's how I can enable TRACE level logging for morphlines? I'm using the CDH 4.4 VM Image.

 

Thanks a lot!

 

Kristof.

Cloudera Employee
Posts: 146
Registered: ‎08-21-2013

Re: How to enable TRACE logging for morphlines

To print diagnostic information, such as the content of records as they pass through the morphline commands, consider enabling TRACE log level. For example, you might add the following lines to your log4j.properties file:

log4j.logger.com.cloudera.cdk.morphline=TRACE

In Cloudera Manager 4 this can be done by navigating to Services -> Flume -> Configuration -> View and Edit -> Agent (Default) -> Advanced -> Agent Logging Safety Valve, followed by a restart of the Flume service.

For HBase Near Real Time Indexing it can be done as described near the bottom of this manual page: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Search/latest/Cloudera-Search-User-Gu...

Wolfgang

New Contributor
Posts: 8
Registered: ‎11-15-2013

Re: How to enable TRACE logging for morphlines

Awesome!
Thank you Wolfgang
Announcements
The Kite SDK is a collection of docs, sample code, APIs, and tools to make Hadoop application development faster. Learn more at http://kitesdk.org.