- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to make Spooling Directory Source work with sub directories too in Apache Flume
- Labels:
-
Apache Flume
Created on ‎08-19-2015 01:59 AM - edited ‎09-16-2022 02:38 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to make Flume Spooling Directory Source work with the Sub Directories of a folder too. My source folder have several other folders too, I want my flume agent to look into these sub directories too for a file to dump it into sink. Is there any way to do it?
Created ‎08-27-2015 01:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There is a patch available FLume-1899, if any body wants he/she can use that for Recursive Directory Search.
Created ‎08-22-2015 08:42 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can create multiple agent sources that point to different sub-directories on the server I believe. Check out the Flume User Guide and search for "Adding multiple flows in an agent" to see if that helps? There's likely even more applicable information for you beyond that section as well.
It also sounds like you may be wanting dynamic recursive directory searching for Flume... I don't know if that's available right now (I don't think it is), but I would be interested if someone knows one way or the other. 🙂
Created ‎08-27-2015 01:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There is a patch available FLume-1899, if any body wants he/she can use that for Recursive Directory Search.
