Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Flume Spooling Directory Source runner has shutdown

avatar
Contributor

Using Flume spooldir source to pull files with Flume 1.5.0-cdh5.3.3 version.

Everything working fine as expected, but log file is just getting bigger and bigger becuase of below info twice per second

 

16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.

 

Not sure if this is Flume issue or I am missing some configuration here.

Please suggest.

1 ACCEPTED SOLUTION

avatar
This was a logging issue in that specific version of CDH. You can either upgrade CDH, or add a logging safety valve for the flume service:
log4j.logger.source.SpoolDirectorySource=WARN

-pd

View solution in original post

3 REPLIES 3

avatar
This was a logging issue in that specific version of CDH. You can either upgrade CDH, or add a logging safety valve for the flume service:
log4j.logger.source.SpoolDirectorySource=WARN

-pd

avatar
Explorer

For whatever reason the string solution for config did not work for me but using

log4j.logger.org.apache.flume.source.SpoolDirectorySource=WARN

did. Enjoy if the first didnt work!

avatar
Explorer

where should we add this warn??