Support Questions

Find answers, ask questions, and share your expertise

How do I create empty HDFS directory through NIFI

avatar
Explorer

Hi all,

I would like to know if it is possible to create empty HDFS directory using NIFI?

Looks like PutHDFS will need a file name. But in my case, I just want to create a directory without putting any file.

1 REPLY 1

avatar
Master Mentor
@David Yu

-

If you are not writing any data in to the new directory, why the need to create it.

NiFi FlowFiles consist of FlowFile attributes and FlowFile content. Each FlowFile will have a filename associated with it. Processors like PutHDFS are designed to operate against incoming FlowFiles and are designed to put a file to HDFS. Without a FlowFile the processor would not be triggered to connect to HDFS at all. It cab be configured to dynamically create missing directories when writing the first file to a new directory path.

-

If you need to create some directory structure on a regular interval (for example new directory each hour based on timestamp), You would need to use a GenerateFlowFile processor (running using cron scheduling strategy and creating a 0 byte File) that feeds a PutHDFS processor. The "Success" relationship from putHDFS should then feed to DeleteHDFS to remove the 0 byte FlowFile that was added to the newly created directory in HDFS.

-

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.