Increase Flume graceful restart time

Ji ZHANG — Fri, 16 Sep 2022 12:12:51 GMT

I'm using Flume's HDFS sink and it sometimes doesn't close the temporary files in HDFS, so that these data become lost. I think it is because supervisord doesn't give Flume agents enough time to stop, since the default stopwaitsecs is 30 seconds, and after that supervisord just "kill -9" it.

WARN killing '388-flume-AGENT' (7994) with SIGKILL

If I kill the process in terminal, it'll take a minute or so to fully stop, thus making no unrenamed temporary files. I'm wondering whether it's possible to increase the stopwaitsecs. Thanks.

Re: Increase Flume graceful restart time

mathieu.d — Fri, 08 Sep 2017 09:50:08 GMT

I believe this wait time of 30s is hard coded into the cloudera agent.

I don't think we can alter it other than doing a real dirty modification which I wouldn't recommend.

regards,

Mathieu

question Re: Increase Flume graceful restart time in Archives of Support Questions (Read Only)

Increase Flume graceful restart time

Re: Increase Flume graceful restart time