Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

yarn can read/write to hdfs, but cannot execute file on hdfs( execute permission is given through Ranger

Re: yarn can read/write to hdfs, but cannot execute file on hdfs( execute permission is given through Ranger

New Contributor

by the way, how could I make the job run as 'admin' other than yarn?, as admin is the user submitted the job

Re: yarn can read/write to hdfs, but cannot execute file on hdfs( execute permission is given through Ranger

New Contributor

I checked out Ranger source code, I saw that Ranger would log some useful information at DEBUG level, do you know how to enable DEBUG level for ranger hdfs-agent, and where to find the log? in namenode log? or in oozie job log?

Re: yarn can read/write to hdfs, but cannot execute file on hdfs( execute permission is given through Ranger

Guru

Hello @Jinyu Li,

For Ranger hdfs-agent debug, please change log4j for NameNode and add this:

log4j.logger.org.apache.ranger=DEBUG

Your debug log messages will appear in NameNode log.

Hope this helps !

Highlighted

Re: yarn can read/write to hdfs, but cannot execute file on hdfs( execute permission is given through Ranger

Expert Contributor

Hi @Jinyu Li

your issue is likely produced by Hive Permission Inheritance.

After creating the tables, the Sqoop app tries to change the owner/mode of the created HDFS files.

Ranger permissions (even rwx) do not give rights to change POSIX owner/mode, which is why the operation fails. Such failure is classified as "EXECUTE" action by Ranger. You can find more details in the HDFS Audit log, stored locally on the NameNode.

Solution: Could you please try to set "hive.warehouse.subdir.inherit.perms" to false and re-run the job? This stops Hive Imports from trying to set permissions, which is fine when Ranger is the primary source of authorization.

see https://cwiki.apache.org/confluence/display/Hive/Permission+Inheritance+in+Hive for more details.

Best, Benjamin