Support Questions
Find answers, ask questions, and share your expertise

[RESOLVED] [FALCON] error during feed retention delete

Explorer

Hello,

During feed replication (replication hdfs between clusterA and clusterB was done correctly), there's issue with FALCON_FEED_RETENTION concerning Path /tmp/falcon/next-vers-current, this path exists on clusterA and clusterB.

Scheduled job :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="next-vers-current" description="next-vers-current" xmlns="uri:falcon:feed:0.1">
 <frequency>hours(6)</frequency>
 <timezone>UTC</timezone>
 <clusters>
 <cluster name="next-rec-cluster" type="source">
 <validity start="2016-05-01T12:00Z" end="2016-05-27T23:00Z"/>
 <retention limit="days(2)" action="delete"/>
 <locations>
 <location type="data" path="/tmp/falcon/next-vers-current"/>
 </locations>
 </cluster>
 <cluster name="current-rec-cluster" type="target">
 <validity start="2016-05-01T12:00Z" end="2016-05-27T23:00Z"/>
 <retention limit="days(2)" action="delete"/>
 <locations>
 <location type="data" path="/tmp/falcon/next-vers-current"/>
 </locations>
 </cluster>
 </clusters>
 <locations>
 <location type="data" path="/tmp/falcon/"/>
 <location type="stats" path="/none"/>
 <location type="meta" path="/none"/>
 </locations>
 <ACL owner="falcon" group="hadoop" permission="0755"/>
 <schema location="/none" provider="none"/>
 <properties><property name="queueName" value="oozie-launcher"/></properties>
</feed>

4531-falcon-feed-retention-01.png

2016-05-25 15:54:15,369  WARN JavaActionExecutor:523 - SERVER[clusterA] USER[falcon] GROUP[-] TOKEN[] APP[FALCON_FEED_RETENTION_next-vers-current] JOB[0000002-160525113126752-oozie-oozi-W] ACTION[0000002-160525113126752-oozie-oozi-W@eviction] Launcher exception: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem
org.apache.oozie.action.hadoop.JavaMainException: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem
    at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:59)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
    at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:35)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem
    at org.apache.falcon.entity.FileSystemStorage.evict(FileSystemStorage.java:306)
    at org.apache.falcon.retention.FeedEvictor.run(FeedEvictor.java:76)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
    at org.apache.falcon.retention.FeedEvictor.main(FeedEvictor.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:56)
    ... 15 more
Caused by: java.io.IOException: Unable to resolve pattern for feedPath: /tmp/falcon/next-vers-current
    at org.apache.falcon.entity.FeedHelper.getFeedBasePath(FeedHelper.java:442)
    at org.apache.falcon.entity.FileSystemStorage.fileSystemEvictor(FileSystemStorage.java:331)
    at org.apache.falcon.entity.FileSystemStorage.evict(FileSystemStorage.java:300)
    ... 23 more
1 ACCEPTED SOLUTION

Accepted Solutions

Re: [RESOLVED] [FALCON] error during feed retention delete

Try to set you location to:

<locationtype="data"path="/tmp/falcon/next-vers-current/${YEAR}/${MONTH}/${DAY}/${HOUR}"/>

From the feed specification page: The granularity of date pattern in the path should be at least that of a frequency of a feed.

View solution in original post

2 REPLIES 2

Re: [RESOLVED] [FALCON] error during feed retention delete

Try to set you location to:

<locationtype="data"path="/tmp/falcon/next-vers-current/${YEAR}/${MONTH}/${DAY}/${HOUR}"/>

From the feed specification page: The granularity of date pattern in the path should be at least that of a frequency of a feed.

View solution in original post

Re: [RESOLVED] [FALCON] error during feed retention delete

Hi @mayki wogno, I see that you marked your question as "Resolved". If my answer below helped you can you please accept it. If you resolved your issue by other means please publish them, and we'll accept your answer. Instead of marking questions as "resolved" we consider them resolved if they are accepted. Tnx!