Increase Flume graceful restart time

[#1]

I'm using Flume's HDFS sink and it sometimes doesn't close the temporary files in HDFS, so that these data become lost. I think it is because supervisord doesn't give Flume agents enough time to stop, since the default stopwaitsecs is 30 seconds, and after that supervisord just "kill -9" it.

WARN killing '388-flume-AGENT' (7994) with SIGKILL

If I kill the process in terminal, it'll take a minute or so to fully stop, thus making no unrenamed temporary files. I'm wondering whether it's possible to increase the stopwaitsecs. Thanks.

[#2 · Accepted solution]

I believe this wait time of 30s is hard coded into the cloudera agent.

I don't think we can alter it other than doing a real dirty modification which I wouldn't recommend.

regards,

Mathieu