Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NIFI - save file with date of file tittle

Solved Go to solution

NIFI - save file with date of file tittle

New Contributor

Hello,

I have a dataflow that give a files to ftp and save thats files into HDFS. The problem is the name of folders in HDFS is the date that the name file contains, for example:

test_20180926.txt --> HDFS: /2018/09/26/test_20180926.txt

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

Re: NIFI - save file with date of file tittle

Master Guru

@Pepelu Rico

The FTP processors (listFTP or GetFTP) create an attribute (filename) on the created NiFi FlowFiles they output. You can use the UpdateAttribute processor and NiFi Expression Language to manipulate the filename string to create a new FlowFile attribute with the desired path you want to they use in your PutHDFS processor.

91564-screen-shot-2018-09-26-at-55931-pm.png

Here you can see the configuration of the UpdateAttribute processor. I have added a "new property" and named it "hdfsPath". This property name will become the name of my new FlowFileAttribute. I then created a NiFi Expression Language statement that extracts the date string from the existing "filename" attribute and converts it in to a path format using:

${filename:substringAfterLast('_'):substringBeforeLast('.'):toDate('yyyyMMdd'):format('/yyy/MM/dd')}

The result of above is assigned to "hdfsPath".

I can then use that new attribute as my path in the PutHDFS processor for example:

91565-screen-shot-2018-09-26-at-60549-pm.png

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

2 REPLIES 2

Re: NIFI - save file with date of file tittle

Master Guru

@Pepelu Rico

The FTP processors (listFTP or GetFTP) create an attribute (filename) on the created NiFi FlowFiles they output. You can use the UpdateAttribute processor and NiFi Expression Language to manipulate the filename string to create a new FlowFile attribute with the desired path you want to they use in your PutHDFS processor.

91564-screen-shot-2018-09-26-at-55931-pm.png

Here you can see the configuration of the UpdateAttribute processor. I have added a "new property" and named it "hdfsPath". This property name will become the name of my new FlowFileAttribute. I then created a NiFi Expression Language statement that extracts the date string from the existing "filename" attribute and converts it in to a path format using:

${filename:substringAfterLast('_'):substringBeforeLast('.'):toDate('yyyyMMdd'):format('/yyy/MM/dd')}

The result of above is assigned to "hdfsPath".

I can then use that new attribute as my path in the PutHDFS processor for example:

91565-screen-shot-2018-09-26-at-60549-pm.png

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

Highlighted

Re: NIFI - save file with date of file tittle

New Contributor

Thank you very much I have to solve it :)