How to make Spooling Directory Source work with sub directories too in Apache Flume

[#1]

How to make Flume Spooling Directory Source work with the Sub Directories of a folder too. My source folder have several other folders too, I want my flume agent to look into these sub directories too for a file to dump it into sink. Is there any way to do it?

[#2]

You can create multiple agent sources that point to different sub-directories on the server I believe.  Check out the Flume User Guide and search for "Adding multiple flows in an agent" to see if that helps?  There's likely even more applicable information for you beyond that section as well.

It also sounds like you may be wanting dynamic recursive directory searching for Flume... I don't know if that's available right now (I don't think it is), but I would be interested if someone knows one way or the other. 🙂

[#3 · Accepted solution]

There is a patch available FLume-1899, if any body wants he/she can use that for Recursive Directory Search.