Reply
Contributor
Posts: 29
Registered: ‎05-11-2016
Accepted Solution

Flume Spooling Directory Source runner has shutdown

Using Flume spooldir source to pull files with Flume 1.5.0-cdh5.3.3 version.

Everything working fine as expected, but log file is just getting bigger and bigger becuase of below info twice per second

 

16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.

 

Not sure if this is Flume issue or I am missing some configuration here.

Please suggest.

Cloudera Employee
Posts: 275
Registered: ‎01-09-2014

Re: Flume Spooling Directory Source runner has shutdown

This was a logging issue in that specific version of CDH. You can either upgrade CDH, or add a logging safety valve for the flume service:
log4j.logger.source.SpoolDirectorySource=WARN

-pd
DBG
New Contributor
Posts: 4
Registered: ‎07-24-2014

Re: Flume Spooling Directory Source runner has shutdown

For whatever reason the string solution for config did not work for me but using

log4j.logger.org.apache.flume.source.SpoolDirectorySource=WARN

did. Enjoy if the first didnt work!

New Contributor
Posts: 1
Registered: ‎01-16-2018

Re: Flume Spooling Directory Source runner has shutdown

where should we add this warn??

Announcements
New solutions