Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Falcon clusters with multiple users

avatar
Master Guru

I want to schedule falcon jobs with different service accounts. user1 with process1 and user2 with process2. But they should use the same cluster.

However this does not work because falcon creates a folder "process" under the staging directory with user1 and a subfolder process1. When I try to submit a process with user2 it fails because it cannot create the folder process2 under the same folder.

Is this actually the supposed behavior? Do I have to create one cluster each for every user who wants to submit falcon jobs pointing to different staging folders? That sounds pretty inconvenient

1 ACCEPTED SOLUTION

avatar

Benjamin Leonhardi: You may be hitting this issue FALCON-1647. What HDP version are you using? Thanks!

View solution in original post

7 REPLIES 7

avatar
Master Guru

I changed the staging/falcon folder to belong to a group of both users. And changed its access rights to 775, this seems to work until now.

avatar

Benjamin Leonhardi: You need to provide correct permissions to staging and working directories. Please use falcon tutorial for reference.

avatar
Master Guru

I did. The subfolder falcon creates however is created with the user I created the first process with. So a second user does not work anymore.

avatar

Benjamin Leonhardi: Can you please provide the error you are getting? Also, can you attach all your entity definitions?

Also, please paste the output of below command. Thanks!

hadoop fs -ls -R /apps/falcon

avatar

Benjamin Leonhardi: You may be hitting this issue FALCON-1647. What HDP version are you using? Thanks!

avatar
Master Guru

Most likely was this. Sorry for not accepting for so long but when I changed the <falconfolder>/staging/falcon folder manually it all worked and I forgot about it Thanks a lot.

https://issues.apache.org/jira/browse/FALCON-1647

avatar
Master Mentor

@Benjamin Leonhardi has this been resolved? I see three replies below and you didn't accept any.