Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi FetchFile processor - Process specific filenames from ListFile (aka File to fetch Property)

Highlighted

NiFi FetchFile processor - Process specific filenames from ListFile (aka File to fetch Property)

Explorer

Hi all.

Just playing around with NiFi to evaluate it.

I'm trying to read files from the same directory and route them to the appropriate section using a FetchFile processor.

Everything is fine when I change ${filename} attribute with the exact file name but I can't filter files evaluating their name.

I can't figure out how to fetch only files starting with two characters (eg. 10,11,12,13)

Any attempt bring me to an error or to an output filename named true

 

My NiFi Flow:

 

ListFile -> FetchFile -> PutHDFS

 

Thanks

1 REPLY 1

Re: NiFi FetchFile processor - Process specific filenames from ListFile (aka File to fetch Property)

Master Guru

@svasi 

 

Sounds like you should be using a RouteOnAttribute processor between your listFile and FetchFile processor so that only those FlowFIles you want to Fetch the content for are routed to the FetchFile processor.

The listFile processor produces one FlowFile for each file that is returned during the listing execution.  The listFile processor writes a number of FlowFile attributes to the FlowFiles that are produced.

Screen Shot 2019-12-23 at 12.26.43 PM.png
The Filename attribute is what you will want to use when writing your NiFi Expression Language (EL) statement for your dynamically created properties added to the RouteOnAttribute processor.

You can find a EL guide under "Help" under the NiFi global menu in the upper right corner of the NiFi UI.

 

Hope this helps,

Matt

Don't have an account?
Coming from Hortonworks? Activate your account here