Support Questions

Find answers, ask questions, and share your expertise

NIFI - save file with date of file tittle

Explorer

Hello,

I have a dataflow that give a files to ftp and save thats files into HDFS. The problem is the name of folders in HDFS is the date that the name file contains, for example:

test_20180926.txt --> HDFS: /2018/09/26/test_20180926.txt

Thanks

1 ACCEPTED SOLUTION

Master Guru

@Pepelu Rico

The FTP processors (listFTP or GetFTP) create an attribute (filename) on the created NiFi FlowFiles they output. You can use the UpdateAttribute processor and NiFi Expression Language to manipulate the filename string to create a new FlowFile attribute with the desired path you want to they use in your PutHDFS processor.

91564-screen-shot-2018-09-26-at-55931-pm.png

Here you can see the configuration of the UpdateAttribute processor. I have added a "new property" and named it "hdfsPath". This property name will become the name of my new FlowFileAttribute. I then created a NiFi Expression Language statement that extracts the date string from the existing "filename" attribute and converts it in to a path format using:

${filename:substringAfterLast('_'):substringBeforeLast('.'):toDate('yyyyMMdd'):format('/yyy/MM/dd')}

The result of above is assigned to "hdfsPath".

I can then use that new attribute as my path in the PutHDFS processor for example:

91565-screen-shot-2018-09-26-at-60549-pm.png

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

View solution in original post

2 REPLIES 2

Master Guru

@Pepelu Rico

The FTP processors (listFTP or GetFTP) create an attribute (filename) on the created NiFi FlowFiles they output. You can use the UpdateAttribute processor and NiFi Expression Language to manipulate the filename string to create a new FlowFile attribute with the desired path you want to they use in your PutHDFS processor.

91564-screen-shot-2018-09-26-at-55931-pm.png

Here you can see the configuration of the UpdateAttribute processor. I have added a "new property" and named it "hdfsPath". This property name will become the name of my new FlowFileAttribute. I then created a NiFi Expression Language statement that extracts the date string from the existing "filename" attribute and converts it in to a path format using:

${filename:substringAfterLast('_'):substringBeforeLast('.'):toDate('yyyyMMdd'):format('/yyy/MM/dd')}

The result of above is assigned to "hdfsPath".

I can then use that new attribute as my path in the PutHDFS processor for example:

91565-screen-shot-2018-09-26-at-60549-pm.png

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

Explorer

Thank you very much I have to solve it 🙂

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.