Support Questions
Find answers, ask questions, and share your expertise

Could not submit falcon feed schedule with different user

Solved Go to solution
Highlighted

Could not submit falcon feed schedule with different user

Rising Star

Hi,

We are using falcon-0.6.

I am trying to submit Falcon feed schedule as user1 but it throws below error

Permission denied: user=user1, access=WRITE, inode="/apps/falcon/backupCluster/staging/falcon/workflows/feed":falcon:hdfs:drwxr-xr-x

I gave 777 permissions to staging directory but I still see the same error.

I am able to submit the schedule as falcon user but could not as any other user.

Please help me with this

Thanks,

Venkat

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Could not submit falcon feed schedule with different user

Explorer

This is a known bug in Falcon and was recently fixed in FALCON-1647 (https://issues.apache.org/jira/browse/FALCON-1647). To fixed your issue, please gave 777 permission to the following two directories: /apps/falcon/backupCluster/staging/falcon/workflows/feed and /apps/falcon/backupCluster/staging/falcon/workflows/process. Otherwise, it only allows the first user to submit entities. Thanks.

View solution in original post

11 REPLIES 11
Highlighted

Re: Could not submit falcon feed schedule with different user

Rising Star

Hi this is a known bug https://issues.apache.org/jira/browse/FALCON-1647. The workaround until we release next version of Falcon is to give 777 permissions to all directories under /apps/falcon/backupCluster/staging/falcon.

Highlighted

Re: Could not submit falcon feed schedule with different user

Explorer

This is a known bug in Falcon and was recently fixed in FALCON-1647 (https://issues.apache.org/jira/browse/FALCON-1647). To fixed your issue, please gave 777 permission to the following two directories: /apps/falcon/backupCluster/staging/falcon/workflows/feed and /apps/falcon/backupCluster/staging/falcon/workflows/process. Otherwise, it only allows the first user to submit entities. Thanks.

View solution in original post

Highlighted

Re: Could not submit falcon feed schedule with different user

Rising Star

Thank you for your quick response.

I am seeing that error even after giving 777 permission to all directories within staging directory for both primaryCluster and backupCluster.

Highlighted

Re: Could not submit falcon feed schedule with different user

Rising Star

Hi Venkata, Can you please go to the falcon.application.log file, and paste the full exception you see when you attempt to submit the feed? This way, we can debug better.

Re: Could not submit falcon feed schedule with different user

Explorer
My falcon.application.log file shows the following exception : 
Caused by: org.apache.falcon.FalconException: E0501 : E0501: Could not perform authorization operation, Unauthorized connection for super-user: oozie from IP 52.8.144.45
	at org.apache.falcon.workflow.engine.OozieWorkflowEngine.dryRunInternal(OozieWorkflowEngine.java:234)
	at org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:172)
	at org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:94)
What & How to configure oozie for falcon ??
Highlighted

Re: Could not submit falcon feed schedule with different user

Rising Star

T//entities/schedule/feed/hdfspdrrep] ~ Unable to schedule workflow (AbstractSchedulableEntityManager:66) org.apache.falcon.FalconException: Error preparing base staging dirs: /apps/falcon/backupCluster/staging/falcon/workflows/feed/hdfspdrrep at org.apache.falcon.workflow.engine.OozieWorkflowEngine.prepareEntityBuildPath(OozieWorkflowEngine.java:185) at org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:153) at org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:76) at org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:63) at org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:116) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=hdpexeusr, access=WRITE, inode="/apps/falcon/backupCluster/staging/falcon/workflows/feed":falcon:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6795) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6777) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6729) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4495) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4465) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4438) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:830) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:614)

.......................

... 61 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hdpexeusr, access=WRITE, inode="/apps/falcon/backupCluster/staging/falcon/workflows/feed":falcon:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6795) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6777) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6729) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4495) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4465) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4438) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:830) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:614)

......................

2015-12-11 14:22:12,668 ERROR - [1088325169@qtp-1225320031-7009:717f3588-92ea-412b-a666-698ac323b961 hdpexeusr:POST//entities/schedule/feed/hdfspdrrep] ~ Action failed: Bad Request Error: Error preparing base staging dirs: /apps/falcon/backupCluster/staging/falcon/workflows/feed/hdfspdrrep (FalconWebException:68) 2015-12-11 14:22:12,669 INFO - [1088325169@qtp-1225320031-7009:717f3588-92ea-412b-a666-698ac323b961 hdpexeusr:POST//entities/schedule/feed/hdfspdrrep] ~ {Action:schedule, Dimensions:{entityType=feed, colo=*, entityName=hdfspdrrep}, Status: FAILED, Time-taken:232141743 ns} (METRIC:38)

Highlighted

Re: Could not submit falcon feed schedule with different user

Rising Star

Hi, the error clearly says you did not give 777 permissions. Can you please try this command?

hadoop fs -chmod -R 777 /apps/falcon/backupCluster/staging

Highlighted

Re: Could not submit falcon feed schedule with different user

Rising Star

The error says

... 61 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hdpexeusr, access=WRITE, inode="/apps/falcon/backupCluster/staging/falcon/workflows/feed":falcon:hdfs:drwxr-xr-x

It is not sufficient to give perms to just the feed or process dir under /apps/falcon/backupCluster/staging/falcon/workflows , you must have permission 777 for ALL directories under /apps/falcon/backupCluster/staging.

Highlighted

Re: Could not submit falcon feed schedule with different user

Rising Star

Hey Balu,

Thanks for quick responses on the issue. I really appreciate it.

I gave recursive 777 permissions to the entire directory.

drwxrwxrwx - falcon hdfs 0 2015-12-10 15:42 /apps/falcon/backupCluster/staging

I still see the below error