Reply
Expert Contributor
Posts: 69
Registered: ‎11-24-2017

Flume includePattern regular expressions

[ Edited ]

 

I don't understand how includePattern works in Flume. It should accept a regular expression, but the only one that is working to me is the following:

 

 

# accepts only .zip files

agent.sources.src1.includePattern=^.*\.zip$

 

But if for example I want to match only files with format WWWW.DDDDD.zip (e.g. abc.123.zip) the followings does not work:

 

agent.sources.src1.includePattern=^[^\.]+\.[^\.]+\.zip$

agent.sources.src1.includePattern=^\w+\.\d+\.zip$

 

Does anyone know how regex expressions work in Flume?

 

 

Cloudera Employee
Posts: 229
Registered: ‎01-09-2014

Re: Flume includePattern regular expressions

What version of CDH are you running? CDH 5.11.0 and above include FLUME-2911 which does allow for regex patterns.



-pd
Highlighted
Expert Contributor
Posts: 69
Registered: ‎11-24-2017

Re: Flume includePattern regular expressions

cdh 5.13.2, but commons regex patterns don't seem to work

Announcements