Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Debug Morphlines configuration

Debug Morphlines configuration

New Contributor

Hi,

 

I have a Flume configuration that reads file from Spool Dir, transform using Interceptors (with Morphlines) and write to HDFS.

 

I have added debug statements in the Morphlines and been tring to debug the configuration. I am running flume from command line with -Dflume switch (please see below for the command that was used).

 

<<<

flume-ng agent --name Agent1 --conf conf --conf-file $conf -Dflume.root.logger=DEBUG,console

>>>

 

However, I don't see any debug statements that are added in Morhphlines getting reported in the console. All flume logging shows INFO statements but not DEBUG statements.

 

Any guidance or inputs would be greatly appreciated.

 

Thank You!

3 REPLIES 3
Highlighted

Re: Debug Morphlines configuration

Explorer

We're using the standard logging command in morphlines:

  { logInfo { format : "output record: {}", args : ["@{}"] } }

 

Which will appear in your /var/flume-ng/<log file>

 

Have you tried that or the logDebug command?  http://kitesdk.org/docs/current/morphlines/morphlines-reference-guide.html#logTrace

 

David

 

Re: Debug Morphlines configuration

Super Collaborator

The best thing to do to fully debug morphlines and see the data record prior to and after every morphline command is to turn on morphline log4j trace, by adding the following to the flume log4j.properties file in your "conf" directory:

log4j.logger.org.kitesdk.morphline=TRACE

You will then see the flow of data through each command in the morphline.

Re: Debug Morphlines configuration

New Contributor
Thank you David and pdvorak for your inputs! I will give it a shot tomorrow but sounds like that will address my issue.