- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
[RESOLVED] [FALCON] error during feed retention delete
- Labels:
-
Apache Falcon
Created on ‎05-25-2016 02:50 PM - edited ‎08-18-2019 04:07 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
During feed replication (replication hdfs between clusterA and clusterB was done correctly), there's issue with FALCON_FEED_RETENTION concerning Path /tmp/falcon/next-vers-current, this path exists on clusterA and clusterB.
Scheduled job :
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <feed name="next-vers-current" description="next-vers-current" xmlns="uri:falcon:feed:0.1"> <frequency>hours(6)</frequency> <timezone>UTC</timezone> <clusters> <cluster name="next-rec-cluster" type="source"> <validity start="2016-05-01T12:00Z" end="2016-05-27T23:00Z"/> <retention limit="days(2)" action="delete"/> <locations> <location type="data" path="/tmp/falcon/next-vers-current"/> </locations> </cluster> <cluster name="current-rec-cluster" type="target"> <validity start="2016-05-01T12:00Z" end="2016-05-27T23:00Z"/> <retention limit="days(2)" action="delete"/> <locations> <location type="data" path="/tmp/falcon/next-vers-current"/> </locations> </cluster> </clusters> <locations> <location type="data" path="/tmp/falcon/"/> <location type="stats" path="/none"/> <location type="meta" path="/none"/> </locations> <ACL owner="falcon" group="hadoop" permission="0755"/> <schema location="/none" provider="none"/> <properties><property name="queueName" value="oozie-launcher"/></properties> </feed>
2016-05-25 15:54:15,369 WARN JavaActionExecutor:523 - SERVER[clusterA] USER[falcon] GROUP[-] TOKEN[] APP[FALCON_FEED_RETENTION_next-vers-current] JOB[0000002-160525113126752-oozie-oozi-W] ACTION[0000002-160525113126752-oozie-oozi-W@eviction] Launcher exception: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem org.apache.oozie.action.hadoop.JavaMainException: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:59) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47) at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) Caused by: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem at org.apache.falcon.entity.FileSystemStorage.evict(FileSystemStorage.java:306) at org.apache.falcon.retention.FeedEvictor.run(FeedEvictor.java:76) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.falcon.retention.FeedEvictor.main(FeedEvictor.java:52) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:56) ... 15 more Caused by: java.io.IOException: Unable to resolve pattern for feedPath: /tmp/falcon/next-vers-current at org.apache.falcon.entity.FeedHelper.getFeedBasePath(FeedHelper.java:442) at org.apache.falcon.entity.FileSystemStorage.fileSystemEvictor(FileSystemStorage.java:331) at org.apache.falcon.entity.FileSystemStorage.evict(FileSystemStorage.java:300) ... 23 more
Created ‎05-27-2016 11:48 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try to set you location to:
<locationtype="data"path="/tmp/falcon/next-vers-current/${YEAR}/${MONTH}/${DAY}/${HOUR}"/>
From the feed specification page: The granularity of date pattern in the path should be at least that of a frequency of a feed.
Created ‎05-27-2016 11:48 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try to set you location to:
<locationtype="data"path="/tmp/falcon/next-vers-current/${YEAR}/${MONTH}/${DAY}/${HOUR}"/>
From the feed specification page: The granularity of date pattern in the path should be at least that of a frequency of a feed.
Created ‎05-29-2016 01:43 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @mayki wogno, I see that you marked your question as "Resolved". If my answer below helped you can you please accept it. If you resolved your issue by other means please publish them, and we'll accept your answer. Instead of marking questions as "resolved" we consider them resolved if they are accepted. Tnx!
