Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

issue with the permissions umask property from nifi PutHDFS processor

Highlighted

issue with the permissions umask property from nifi PutHDFS processor

Rising Star

Hi,

When i write a file in hdfs with the nifi processor PutHDFS, the permissions umask property doesn't seem to work. In fact, i set it to "777" and i also change the owner and the group of the file using the properties Remote Owner and Remote Group.

The written file has no permissions but the owner and the group are set as i want them to.

Thank you in advance.

9 REPLIES 9

Re: issue with the permissions umask property from nifi PutHDFS processor

@Lubin Lemarchand

, when you say you set it to 777, are you referring to the umask property? The umask acts as a filter to restrict permissions from newly created directories and files. It does not define the desired outcome for permissions of newly created directories and files.

For example, if the umask is set to 777, then I expect newly created directories and files will end up with permissions of 000. If instead the umask is set to 000, then I expect newly created directories will get 777 and newly created files will get 666. The default value for the umask is 022. In that case, newly created directories get 755 (777 with 022 filtered away = 755). Newly created files get 644 (666 with 022 filtered away = 644).

Does this explain the behavior that you're seeing?

Re: issue with the permissions umask property from nifi PutHDFS processor

Rising Star

@Chris Nauroth

I will check tomorrow but yes it definetly explains what i've seen. I wish it was explained quite as thoroughly in the nifi documentation. Anyway, thank you very much.

Re: issue with the permissions umask property from nifi PutHDFS processor

@Lubin Lemarchand, I'm not familiar with the NiFi documentation, but I know the HDFS documentation discusses file system permissions, including the effects of the umask. http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html

Re: issue with the permissions umask property from nifi PutHDFS processor

Rising Star

Update : I'm writing several Flowfiles in a directory. The Directory property is set to "tmp/myDirectory". I've set the umask to "022" following @Chris Nauroth 's advice but it didn't make any difference. The written files still have no permissions at all and the directory permission don't seem to be affected either.

Re: issue with the permissions umask property from nifi PutHDFS processor

@Lubin Lemarchand, unfortunately, I don't have any further advice. It would be great if someone familiar with NiFi could reply.

Re: issue with the permissions umask property from nifi PutHDFS processor

Super Guru

no matter what you set it, it always is set wrong

Re: issue with the permissions umask property from nifi PutHDFS processor

Mentor

Did you try it in latest NiFi? We're 2-3 versions past that now. another option is to add processor ExecuteProcess to apply permissions following putHDFS

Re: issue with the permissions umask property from nifi PutHDFS processor

New Contributor

can you explain me how to put permissions in the ExecuteProcess ?

i have the problem that the PutHDFS failed writing my NiFi data flow from twitter into hdfs.

https://community.hortonworks.com/questions/86534/sentiment-analysis-empty-tweets-text-table.html#an...

Re: issue with the permissions umask property from nifi PutHDFS processor

Expert Contributor

You should explain why 2-3 versions ago was an issue, as well as an example of this ExecuteProcess you've mentioned in other posts (HERE). This answer has almost no useful information on the deafult NiFi behavior regarding PutHDFS.

Don't have an account?
Coming from Hortonworks? Activate your account here