Support Questions
Find answers, ask questions, and share your expertise

How to make Spooling Directory Source work with sub directories too in Apache Flume

New Contributor

How to make Flume Spooling Directory Source work with the Sub Directories of a folder too. My source folder have several other folders too, I want my flume agent to look into these sub directories too for a file to dump it into sink. Is there any way to do it?

1 ACCEPTED SOLUTION

Accepted Solutions

New Contributor

There is a patch available FLume-1899, if any body wants he/she can use that for Recursive Directory Search.

View solution in original post

2 REPLIES 2

Rising Star

You can create multiple agent sources that point to different sub-directories on the server I believe.  Check out the Flume User Guide and search for "Adding multiple flows in an agent" to see if that helps?  There's likely even more applicable information for you beyond that section as well.

 

It also sounds like you may be wanting dynamic recursive directory searching for Flume... I don't know if that's available right now (I don't think it is), but I would be interested if someone knows one way or the other. 🙂

New Contributor

There is a patch available FLume-1899, if any body wants he/she can use that for Recursive Directory Search.

View solution in original post