Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to provide filenames dynamically in NIFI?

Solved Go to solution
Highlighted

How to provide filenames dynamically in NIFI?

Explorer

Hi Team,

I have a zipped file which contains so many text files. I used unpackContent processor to unzip the files. I want to store those unzipped files in another location. But the file names should be replaced with file1.txt, file2.txt, file3.txt and so on. How can I give file names dynamically? I thought updateAttribute can be used in this. But I'm not sure at the expression language to get this done. Can anyone help me to write a expression language in updateAttribute processor or any alternative processor/solution for this.

 

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to provide filenames dynamically in NIFI?

Master Guru

Hello @girish6,

 

NiFi Expression language does have a function that will increment a number, but it does not reset back to zero (unless you restart NiFi) and is incremented globally across call calls to that function.

 

Screen Shot 2019-10-10 at 12.37.15 PM.png

 

So while this function will give each FlowFile a unique filename per NiFi node, there is no guarantee that they will be sequential or will start at 0 for new source zip file you extract.  If this works for your use case the NiFi Expression language statement would look like this in the UpdateAttribute processor:
Screen Shot 2019-10-10 at 12.44.10 PM.png

If you are just looking for filename uniqueness, a better solution may be to instead append the FlowFile's uniquely assigned UUID to the filename instead as follows:
Screen Shot 2019-10-10 at 12.46.25 PM.png

 

Tips: 
1. There is an expression language guide in the embedded help of your NiFi installation found by clicking on the global menu icon in upper right corner.
2. Within a NiFi processor property that support NiFi EL you can display a list of functions with details by typing "${" (signifies start of an EL statement) and then hold "Control" while clicking space bar.  This also works if you are mid statement and type ":"then "control" + click spacebar.

Examples:
file-${   (then cntrl+spacebar)
${uuid:  (then cntrl+spacebar)

Hope this helps,
Matt

1 REPLY 1

Re: How to provide filenames dynamically in NIFI?

Master Guru

Hello @girish6,

 

NiFi Expression language does have a function that will increment a number, but it does not reset back to zero (unless you restart NiFi) and is incremented globally across call calls to that function.

 

Screen Shot 2019-10-10 at 12.37.15 PM.png

 

So while this function will give each FlowFile a unique filename per NiFi node, there is no guarantee that they will be sequential or will start at 0 for new source zip file you extract.  If this works for your use case the NiFi Expression language statement would look like this in the UpdateAttribute processor:
Screen Shot 2019-10-10 at 12.44.10 PM.png

If you are just looking for filename uniqueness, a better solution may be to instead append the FlowFile's uniquely assigned UUID to the filename instead as follows:
Screen Shot 2019-10-10 at 12.46.25 PM.png

 

Tips: 
1. There is an expression language guide in the embedded help of your NiFi installation found by clicking on the global menu icon in upper right corner.
2. Within a NiFi processor property that support NiFi EL you can display a list of functions with details by typing "${" (signifies start of an EL statement) and then hold "Control" while clicking space bar.  This also works if you are mid statement and type ":"then "control" + click spacebar.

Examples:
file-${   (then cntrl+spacebar)
${uuid:  (then cntrl+spacebar)

Hope this helps,
Matt

Don't have an account?
Coming from Hortonworks? Activate your account here