question Re: Flume Spooling Directory Source runner has shutdown in Archives of Support Questions (Read Only)

Flume Spooling Directory Source runner has shutdown

Vinod369 — Fri, 16 Sep 2022 10:26:00 GMT

Using Flume spooldir source to pull files with Flume 1.5.0-cdh5.3.3 version.

Everything working fine as expected, but log file is just getting bigger and bigger becuase of below info twice per second

16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.

Not sure if this is Flume issue or I am missing some configuration here.

Please suggest.

Re: Flume Spooling Directory Source runner has shutdown

pdvorak — Thu, 23 Jun 2016 20:04:32 GMT

This was a logging issue in that specific version of CDH. You can either upgrade CDH, or add a logging safety valve for the flume service:
log4j.logger.source.SpoolDirectorySource=WARN

-pd

Re: Flume Spooling Directory Source runner has shutdown

DBG — Wed, 18 Jan 2017 17:46:26 GMT

For whatever reason the string solution for config did not work for me but using

log4j.logger.org.apache.flume.source.SpoolDirectorySource=WARN

did. Enjoy if the first didnt work!

Re: Flume Spooling Directory Source runner has shutdown

rp4345 — Wed, 17 Jan 2018 06:01:22 GMT

where should we add this warn??