Support Questions

Find answers, ask questions, and share your expertise

Flume Spooling Directory Source runner has shutdown

Contributor

Using Flume spooldir source to pull files with Flume 1.5.0-cdh5.3.3 version.

Everything working fine as expected, but log file is just getting bigger and bigger becuase of below info twice per second

 

16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.

 

Not sure if this is Flume issue or I am missing some configuration here.

Please suggest.

1 ACCEPTED SOLUTION

Super Collaborator
This was a logging issue in that specific version of CDH. You can either upgrade CDH, or add a logging safety valve for the flume service:
log4j.logger.source.SpoolDirectorySource=WARN

-pd

View solution in original post

3 REPLIES 3

Super Collaborator
This was a logging issue in that specific version of CDH. You can either upgrade CDH, or add a logging safety valve for the flume service:
log4j.logger.source.SpoolDirectorySource=WARN

-pd

New Contributor

For whatever reason the string solution for config did not work for me but using

log4j.logger.org.apache.flume.source.SpoolDirectorySource=WARN

did. Enjoy if the first didnt work!

Explorer

where should we add this warn??

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.