- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Falcon clusters with multiple users
- Labels:
-
Apache Falcon
Created ‎01-28-2016 05:13 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I want to schedule falcon jobs with different service accounts. user1 with process1 and user2 with process2. But they should use the same cluster.
However this does not work because falcon creates a folder "process" under the staging directory with user1 and a subfolder process1. When I try to submit a process with user2 it fails because it cannot create the folder process2 under the same folder.
Is this actually the supposed behavior? Do I have to create one cluster each for every user who wants to submit falcon jobs pointing to different staging folders? That sounds pretty inconvenient
Created ‎01-28-2016 10:20 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Benjamin Leonhardi: You may be hitting this issue FALCON-1647. What HDP version are you using? Thanks!
Created ‎01-28-2016 05:47 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I changed the staging/falcon folder to belong to a group of both users. And changed its access rights to 775, this seems to work until now.
Created ‎01-28-2016 06:32 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Benjamin Leonhardi: You need to provide correct permissions to staging and working directories. Please use falcon tutorial for reference.
Created ‎01-28-2016 08:31 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I did. The subfolder falcon creates however is created with the user I created the first process with. So a second user does not work anymore.
Created ‎01-28-2016 09:30 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Benjamin Leonhardi: Can you please provide the error you are getting? Also, can you attach all your entity definitions?
Also, please paste the output of below command. Thanks!
hadoop fs -ls -R /apps/falcon
Created ‎01-28-2016 10:20 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Benjamin Leonhardi: You may be hitting this issue FALCON-1647. What HDP version are you using? Thanks!
Created ‎02-02-2016 02:35 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Most likely was this. Sorry for not accepting for so long but when I changed the <falconfolder>/staging/falcon folder manually it all worked and I forgot about it Thanks a lot.
Created ‎02-02-2016 02:27 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Benjamin Leonhardi has this been resolved? I see three replies below and you didn't accept any.
