When i write a file in hdfs with the nifi processor PutHDFS, the permissions umask property doesn't seem to work. In fact, i set it to "777" and i also change the owner and the group of the file using the properties Remote Owner and Remote Group.
The written file has no permissions but the owner and the group are set as i want them to.
Thank you in advance.
, when you say you set it to 777, are you referring to the umask property? The umask acts as a filter to restrict permissions from newly created directories and files. It does not define the desired outcome for permissions of newly created directories and files.
For example, if the umask is set to 777, then I expect newly created directories and files will end up with permissions of 000. If instead the umask is set to 000, then I expect newly created directories will get 777 and newly created files will get 666. The default value for the umask is 022. In that case, newly created directories get 755 (777 with 022 filtered away = 755). Newly created files get 644 (666 with 022 filtered away = 644).
Does this explain the behavior that you're seeing?
@Lubin Lemarchand, I'm not familiar with the NiFi documentation, but I know the HDFS documentation discusses file system permissions, including the effects of the umask. http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
Update : I'm writing several Flowfiles in a directory. The Directory property is set to "tmp/myDirectory". I've set the umask to "022" following @Chris Nauroth 's advice but it didn't make any difference. The written files still have no permissions at all and the directory permission don't seem to be affected either.
Did you try it in latest NiFi? We're 2-3 versions past that now. another option is to add processor ExecuteProcess to apply permissions following putHDFS
can you explain me how to put permissions in the ExecuteProcess ?
i have the problem that the PutHDFS failed writing my NiFi data flow from twitter into hdfs.
You should explain why 2-3 versions ago was an issue, as well as an example of this ExecuteProcess you've mentioned in other posts (HERE). This answer has almost no useful information on the deafult NiFi behavior regarding PutHDFS.