NiFi PutHDFS Processor doesn't use "fs.permissions.umask-mode"

[#1]

Hi,

I try to put a file into HDFS via PutHDFS Processor. At least the flow works, but "fs.permissons.umask-mode" on hdfs-site.xml is ignored.
After some investigation, I found PutHDFS Processor refers deprecated property "dfs.umaskmode" which is now replaced by "fs.permissions.umask-mode".
I set "dfs.umaskmode" on hdfs-site.xml by myself, but still the Processor doesn't set permission properly.
"Permission umask" property on PutHDFS processor works, but I don't wanna set it for each Processor.
How can I let the processor use "fs.permissions.umask-mode"? Any workaround will do.

[#2 · Accepted solution]

@Kei Miyauchi

With core-site.xml and hdfs-site.xml being provided in the "Hadoop Configuration Resources" property, that config is passed to the hadoop client that PutHDFS uses to send data to HDFS. However, in the code it looks like if the "Permissions umask" property is not set, then PutHDFS will use a default umask of "18", which is pulled from FsPermission.java from hadoop-common.

Unfortunately, I don't think there's a workaround. The "Permissions umask" property doesn't support EL, so for now you would have to set the umask explicitly via the property.

I created bug NIFI-5575 to track the issue.

[#3]

It's a pity that there's no workaround. Okay now I'll set "Permission umask" for each processor until the issue resolved.
Thank you for the reply and advice. I'll watch the issue you've created.

[#4]

I apologize that I couldn't think of a workaround, and that you'll have to set "Permission umask" for each processor. After NIFI-5575 is resolved, it will be included in a future HDF release, and you should be able to update your flow to remove the specific settings in each processor.