Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Could not submit falcon feed schedule with different user

Rising Star

Hi,

We are using falcon-0.6.

I am trying to submit Falcon feed schedule as user1 but it throws below error

Permission denied: user=user1, access=WRITE, inode="/apps/falcon/backupCluster/staging/falcon/workflows/feed":falcon:hdfs:drwxr-xr-x

I gave 777 permissions to staging directory but I still see the same error.

I am able to submit the schedule as falcon user but could not as any other user.

Please help me with this

Thanks,

Venkat

1 ACCEPTED SOLUTION

Explorer

This is a known bug in Falcon and was recently fixed in FALCON-1647 (https://issues.apache.org/jira/browse/FALCON-1647). To fixed your issue, please gave 777 permission to the following two directories: /apps/falcon/backupCluster/staging/falcon/workflows/feed and /apps/falcon/backupCluster/staging/falcon/workflows/process. Otherwise, it only allows the first user to submit entities. Thanks.

View solution in original post

11 REPLIES 11

Rising Star

Hi this is a known bug https://issues.apache.org/jira/browse/FALCON-1647. The workaround until we release next version of Falcon is to give 777 permissions to all directories under /apps/falcon/backupCluster/staging/falcon.

Explorer

This is a known bug in Falcon and was recently fixed in FALCON-1647 (https://issues.apache.org/jira/browse/FALCON-1647). To fixed your issue, please gave 777 permission to the following two directories: /apps/falcon/backupCluster/staging/falcon/workflows/feed and /apps/falcon/backupCluster/staging/falcon/workflows/process. Otherwise, it only allows the first user to submit entities. Thanks.

Rising Star

Thank you for your quick response.

I am seeing that error even after giving 777 permission to all directories within staging directory for both primaryCluster and backupCluster.

Rising Star

Hi Venkata, Can you please go to the falcon.application.log file, and paste the full exception you see when you attempt to submit the feed? This way, we can debug better.

Explorer
My falcon.application.log file shows the following exception : 
Caused by: org.apache.falcon.FalconException: E0501 : E0501: Could not perform authorization operation, Unauthorized connection for super-user: oozie from IP 52.8.144.45
	at org.apache.falcon.workflow.engine.OozieWorkflowEngine.dryRunInternal(OozieWorkflowEngine.java:234)
	at org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:172)
	at org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:94)
What & How to configure oozie for falcon ??

Rising Star

T//entities/schedule/feed/hdfspdrrep] ~ Unable to schedule workflow (AbstractSchedulableEntityManager:66) org.apache.falcon.FalconException: Error preparing base staging dirs: /apps/falcon/backupCluster/staging/falcon/workflows/feed/hdfspdrrep at org.apache.falcon.workflow.engine.OozieWorkflowEngine.prepareEntityBuildPath(OozieWorkflowEngine.java:185) at org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:153) at org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:76) at org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:63) at org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:116) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=hdpexeusr, access=WRITE, inode="/apps/falcon/backupCluster/staging/falcon/workflows/feed":falcon:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6795) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6777) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6729) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4495) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4465) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4438) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:830) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:614)

.......................

... 61 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hdpexeusr, access=WRITE, inode="/apps/falcon/backupCluster/staging/falcon/workflows/feed":falcon:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6795) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6777) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6729) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4495) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4465) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4438) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:830) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:614)

......................

2015-12-11 14:22:12,668 ERROR - [1088325169@qtp-1225320031-7009:717f3588-92ea-412b-a666-698ac323b961 hdpexeusr:POST//entities/schedule/feed/hdfspdrrep] ~ Action failed: Bad Request Error: Error preparing base staging dirs: /apps/falcon/backupCluster/staging/falcon/workflows/feed/hdfspdrrep (FalconWebException:68) 2015-12-11 14:22:12,669 INFO - [1088325169@qtp-1225320031-7009:717f3588-92ea-412b-a666-698ac323b961 hdpexeusr:POST//entities/schedule/feed/hdfspdrrep] ~ {Action:schedule, Dimensions:{entityType=feed, colo=*, entityName=hdfspdrrep}, Status: FAILED, Time-taken:232141743 ns} (METRIC:38)

Rising Star

Hi, the error clearly says you did not give 777 permissions. Can you please try this command?

hadoop fs -chmod -R 777 /apps/falcon/backupCluster/staging

Rising Star

The error says

... 61 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hdpexeusr, access=WRITE, inode="/apps/falcon/backupCluster/staging/falcon/workflows/feed":falcon:hdfs:drwxr-xr-x

It is not sufficient to give perms to just the feed or process dir under /apps/falcon/backupCluster/staging/falcon/workflows , you must have permission 777 for ALL directories under /apps/falcon/backupCluster/staging.

Rising Star

Hey Balu,

Thanks for quick responses on the issue. I really appreciate it.

I gave recursive 777 permissions to the entire directory.

drwxrwxrwx - falcon hdfs 0 2015-12-10 15:42 /apps/falcon/backupCluster/staging

I still see the below error

Rising Star

I think the problems is, though I gave 777 over /apps/falcon/backupCluster/staging, when I tried creating directory under /apps/falcon/backupCluster/staging/falcon/workflows/feed as falcon user, it is created with 755 permissions as below

[falcon@hostname ~]$ hdfs dfs -mkdir /apps/falcon/backupCluster/staging/falcon/workflows/feed/test1

hdfs dfs -ls /apps/falcon/backupCluster/staging/falcon/workflows/feed

drwxr-xr-x - falcon hdfs 0 2015-12-18 09:55 /apps/falcon/backupCluster/staging/falcon/workflows/feed/test1

The directory is not created with 777 permission (umask 022).

Do you think it could be the reason?

Rising Star

Yes. That is correct.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.