question On Nifi Is it possible using PutHdfs Processor to configure so that files are written to day wise folder ,rolling to new day folder everyday in Archives of Support Questions (Read Only)

On Nifi Is it possible using PutHdfs Processor to configure so that files are written to day wise folder ,rolling to new day folder everyday

ManishaJain — Tue, 05 Sep 2017 18:47:31 GMT

For Ex:

USing getKafka -to read the stream

PutHDFS to write to HDFS

Can it be configured in the processor to create new folders when a new day starts and write the kafka topic data to that folder.

Basically to maintain day wise folders in HDFS using putHDFS processor in nifi.

Re: On Nifi Is it possible using PutHdfs Processor to configure so that files are written to day wise folder ,rolling to new day folder everyday

Shu_ashu — Sun, 18 Aug 2019 09:58:30 GMT

Hi @manisha jain, yep it is possible with PutHDFS processor.
in puthdfs processor keep directory property as

/user/kafka/${now():format("yyyy-MM-dd")}

Expression:-
${now():format("yyyy-MM-dd")}
above expression now will be the current timestamp and extracts only the date and creates the directory with the date dynamically and and stores all the flow files that are generated with same date will be stored into the same directory.

PutHDFS properties:-

Re: On Nifi Is it possible using PutHdfs Processor to configure so that files are written to day wise folder ,rolling to new day folder everyday

ManishaJain — Thu, 07 Sep 2017 11:16:06 GMT

Thanks @Yash i have used the same approach,it works.