Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Flume Spooling Directory Source runner has shutdown

Solved Go to solution

Flume Spooling Directory Source runner has shutdown

Contributor

Using Flume spooldir source to pull files with Flume 1.5.0-cdh5.3.3 version.

Everything working fine as expected, but log file is just getting bigger and bigger becuase of below info twice per second

 

16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:58 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:19:59 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.
16/06/17 09:20:00 INFO source.SpoolDirectorySource: Spooling Directory Source runner has shutdown.

 

Not sure if this is Flume issue or I am missing some configuration here.

Please suggest.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Flume Spooling Directory Source runner has shutdown

Super Collaborator
This was a logging issue in that specific version of CDH. You can either upgrade CDH, or add a logging safety valve for the flume service:
log4j.logger.source.SpoolDirectorySource=WARN

-pd
3 REPLIES 3

Re: Flume Spooling Directory Source runner has shutdown

Super Collaborator
This was a logging issue in that specific version of CDH. You can either upgrade CDH, or add a logging safety valve for the flume service:
log4j.logger.source.SpoolDirectorySource=WARN

-pd

Re: Flume Spooling Directory Source runner has shutdown

New Contributor

For whatever reason the string solution for config did not work for me but using

log4j.logger.org.apache.flume.source.SpoolDirectorySource=WARN

did. Enjoy if the first didnt work!

Re: Flume Spooling Directory Source runner has shutdown

New Contributor

where should we add this warn??

Don't have an account?
Coming from Hortonworks? Activate your account here