Member since
09-23-2015
800
Posts
898
Kudos Received
185
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5410 | 08-12-2016 01:02 PM | |
2203 | 08-08-2016 10:00 AM | |
2612 | 08-03-2016 04:44 PM | |
5501 | 08-03-2016 02:53 PM | |
1424 | 08-01-2016 02:38 PM |
01-28-2016
08:31 PM
I did. The subfolder falcon creates however is created with the user I created the first process with. So a second user does not work anymore.
... View more
01-28-2016
06:30 PM
1 Kudo
Ah cool didn't know that one. I suppose this is only honored if you use CombinedFileInputFormat and its child classes? I am pretty sure the standard FileInputFormat uses one file at most.
... View more
01-28-2016
05:47 PM
I changed the staging/falcon folder to belong to a group of both users. And changed its access rights to 775, this seems to work until now.
... View more
01-28-2016
05:42 PM
1 Kudo
Perhaps some error in the pig script? You will find the logs in the logs of the oozie launcher action ( the map task log ) or the pig action that gets spawned off. Hue is convenient to get the logs or you can go directly to the resourcemanager UI. When a process is executed you will have one job that is the launcher it will contain the parameters for the pig script and any error that is returned by the pig command. You will have a second job that is the actual pig execution. You should find the problems in one or the other. If these jobs don't exist you can also go to the oozie ui and see why these actions are not spawned off.
... View more
01-28-2016
05:39 PM
2 Kudos
In the partition field the left value is the column so you do not change this. The right value is the partition you want to load into. I.e. INSERT ... partition( TRAN_DATE= your date ) My tip: change the partition field from Date to Integer. Then it would work. Alternatively you could cast the right side to a date. partition ( TRAN_DATE = to_date(${day},'YYYY-MM-DD')) but honestly a partition column of string or integer is most likely less headache. I never used a Date so not sure if this would work.
... View more
01-28-2016
05:13 PM
I want to schedule falcon jobs with different service accounts. user1 with process1 and user2 with process2. But they should use the same cluster. However this does not work because falcon creates a folder "process" under the staging directory with user1 and a subfolder process1. When I try to submit a process with user2 it fails because it cannot create the folder process2 under the same folder. Is this actually the supposed behavior? Do I have to create one cluster each for every user who wants to submit falcon jobs pointing to different staging folders? That sounds pretty inconvenient
... View more
Labels:
- Labels:
-
Apache Falcon
01-28-2016
05:06 PM
And a falcon restart fixed that. The retention cleanup now works. Thanks a lot
... View more
01-28-2016
04:54 PM
Still didn't work I get now some kerberos errors when trying to schedule the feed but at least I know wht the retention does not work. Thanks.
... View more
01-28-2016
04:50 PM
aaah I didn't know that I only scheduled the process using the feeds. Let me try that
... View more
01-28-2016
02:33 PM
2 Kudos
It should work even without Ranger. Are you sure you restarted HDFS and did everything correctly? You have to add it to the custom properties as you did. I used it recently in the certification and it worked for me. http://hortonworks.com/blog/hdfs-acls-fine-grained-permissions-hdfs-files-hadoop/
... View more