Created on 05-05-2016 04:01 AM - edited 08-19-2019 02:21 AM
Following the "Processing Data Pipeline with Apache Falcon" tutorial and getting below errors in Oozie after scheduling the processes.
Created 05-26-2016 07:54 AM
Got same issue .. :-(
Created 05-26-2016 03:21 PM
and @mayki wogno : What does your logs in Yarn say? Do you see an error similar to this?
Caused by: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem at org.apache.falcon.entity.FileSystemStorage.evict(FileSystemStorage.java:306) at org.apache.falcon.retention.FeedEvictor.run(FeedEvictor.java:76) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.falcon.retention.FeedEvictor.main(FeedEvictor.java:52) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:56) ... 15 more Caused by: java.io.IOException: Unable to resolve pattern for feedPath: / at org.apache.falcon.entity.FeedHelper.getFeedBasePath(FeedHelper.java:518) at org.apache.falcon.entity.FileSystemStorage.fileSystemEvictor(FileSystemStorage.java:331) at org.apache.falcon.entity.FileSystemStorage.evict(FileSystemStorage.java:300) ... 23 more
Created 05-26-2016 03:23 PM
You can find logs by clicking on console url and clicking on the "1" under map tasks. I request you to share the hadoop version you are using.
Created 05-26-2016 03:31 PM
it seems that the location data path must be like :
<location type="data" path="/tmp/demo-falcon/process/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
Created 05-27-2016 11:43 PM
From the Feed Specification page: A location tag specifies the type of location like data, meta, stats and the corresponding paths for them. A feed should at least define the location for type data, which specifies the HDFS path pattern where the feed is generated periodically. ex: type="data" path="/projects/TrafficHourly/${YEAR}-${MONTH}-${DAY}/traffic" The granularity of date pattern in the path should be at least that of a frequency of a feed.
So if your frequency is for example 1 hour, you need also ${HOUR} but no ${MINUTE}
Created 06-08-2016 12:02 PM
What is the answer to this problem ??
Created 06-08-2016 03:26 PM
Apart from defining location for data, can you please try setting "stats" and "meta" to "/tmp/${YEAR}-${MONTH}-${DAY}" ? Please let us know if this works. Also, please let me know the Java version you are using, I suspect the change in behavior is due to possible change in how pattern matcher behaves between Java 7 and 8.