question Re: NiFi PutHDFS & create missing directories in Archives of Support Questions (Read Only)

NiFi PutHDFS & create missing directories

Raj_B — Thu, 12 Jan 2017 03:20:43 GMT

Hello,

From using the NiFi's PutHDFS processor, it seems it creates missing directories in HDFS by default, but documentation doesn't specify whether it would or would not create missing directories and there are no properties related to it (like there is one for PutFile I believe). Does anyone know anything more about this "undocumented" feature, if I can call that.

My only concern is, if this is not a documented feature, does this feature (creating missing directories) work reliably in a Production environment.

Did i miss this in the documentation ?

Thanks.

Re: NiFi PutHDFS & create missing directories

bbende — Thu, 12 Jan 2017 22:50:35 GMT

PutHDFS will always create the directory if it does not exist:

https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L260-L270

I don't see any concerns about this working any differently in production.

Re: NiFi PutHDFS & create missing directories

Raj_B — Thu, 12 Jan 2017 23:00:13 GMT

Thanks @Bryan Bende for confirming, good to know; since I didn't see it in the documentation, I was concerned.