Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Increase Flume graceful restart time

SOLVED Go to solution

Increase Flume graceful restart time

New Contributor

I'm using Flume's HDFS sink and it sometimes doesn't close the temporary files in HDFS, so that these data become lost. I think it is because supervisord doesn't give Flume agents enough time to stop, since the default stopwaitsecs is 30 seconds, and after that supervisord just "kill -9" it.

 

WARN killing '388-flume-AGENT' (7994) with SIGKILL

 

If I kill the process in terminal, it'll take a minute or so to fully stop, thus making no unrenamed temporary files. I'm wondering whether it's possible to increase the stopwaitsecs. Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Increase Flume graceful restart time

Super Collaborator

I believe this wait time of 30s is hard coded into the cloudera agent.

I don't think we can alter it other than doing a real dirty modification which I wouldn't recommend.

 

regards,

Mathieu

 

1 REPLY 1

Re: Increase Flume graceful restart time

Super Collaborator

I believe this wait time of 30s is hard coded into the cloudera agent.

I don't think we can alter it other than doing a real dirty modification which I wouldn't recommend.

 

regards,

Mathieu