New Contributor
Posts: 2
Registered: ‎06-10-2015

Debug Morphlines configuration



I have a Flume configuration that reads file from Spool Dir, transform using Interceptors (with Morphlines) and write to HDFS.


I have added debug statements in the Morphlines and been tring to debug the configuration. I am running flume from command line with -Dflume switch (please see below for the command that was used).



flume-ng agent --name Agent1 --conf conf --conf-file $conf -Dflume.root.logger=DEBUG,console



However, I don't see any debug statements that are added in Morhphlines getting reported in the console. All flume logging shows INFO statements but not DEBUG statements.


Any guidance or inputs would be greatly appreciated.


Thank You!

Posts: 11
Registered: ‎06-05-2015

Re: Debug Morphlines configuration

[ Edited ]

We're using the standard logging command in morphlines:

  { logInfo { format : "output record: {}", args : ["@{}"] } }


Which will appear in your /var/flume-ng/<log file>


Have you tried that or the logDebug command?




Cloudera Employee
Posts: 277
Registered: ‎01-09-2014

Re: Debug Morphlines configuration

[ Edited ]

The best thing to do to fully debug morphlines and see the data record prior to and after every morphline command is to turn on morphline log4j trace, by adding the following to the flume file in your "conf" directory:

You will then see the flow of data through each command in the morphline.

New Contributor
Posts: 2
Registered: ‎06-10-2015

Re: Debug Morphlines configuration

Thank you David and pdvorak for your inputs! I will give it a shot tomorrow but sounds like that will address my issue.