Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How do I create empty HDFS directory through NIFI

How do I create empty HDFS directory through NIFI

New Contributor

Hi all,

I would like to know if it is possible to create empty HDFS directory using NIFI?

Looks like PutHDFS will need a file name. But in my case, I just want to create a directory without putting any file.

1 REPLY 1

Re: How do I create empty HDFS directory through NIFI

Master Guru
@David Yu

-

If you are not writing any data in to the new directory, why the need to create it.

NiFi FlowFiles consist of FlowFile attributes and FlowFile content. Each FlowFile will have a filename associated with it. Processors like PutHDFS are designed to operate against incoming FlowFiles and are designed to put a file to HDFS. Without a FlowFile the processor would not be triggered to connect to HDFS at all. It cab be configured to dynamically create missing directories when writing the first file to a new directory path.

-

If you need to create some directory structure on a regular interval (for example new directory each hour based on timestamp), You would need to use a GenerateFlowFile processor (running using cron scheduling strategy and creating a 0 byte File) that feeds a PutHDFS processor. The "Success" relationship from putHDFS should then feed to DeleteHDFS to remove the 0 byte FlowFile that was added to the newly created directory in HDFS.

-

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.