Support Questions
Find answers, ask questions, and share your expertise

org.apache.falcon.FalconException: Couldn't evict feed from fileSystem (JA018)

Explorer

Following the "Processing Data Pipeline with Apache Falcon" tutorial and getting below errors in Oozie after scheduling the processes.

3970-screen-shot-2016-05-04-at-122854-pm.png

3971-screen-shot-2016-05-04-at-122814-pm.png

7 REPLIES 7

Re: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem (JA018)

Explorer

Got same issue .. 😞

Re: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem (JA018)

Rising Star

@Junaid Rao

and @mayki wogno : What does your logs in Yarn say? Do you see an error similar to this?

Caused by: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem
	at org.apache.falcon.entity.FileSystemStorage.evict(FileSystemStorage.java:306)
	at org.apache.falcon.retention.FeedEvictor.run(FeedEvictor.java:76)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.falcon.retention.FeedEvictor.main(FeedEvictor.java:52)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:56)
	... 15 more
Caused by: java.io.IOException: Unable to resolve pattern for feedPath: /
	at org.apache.falcon.entity.FeedHelper.getFeedBasePath(FeedHelper.java:518)
	at org.apache.falcon.entity.FileSystemStorage.fileSystemEvictor(FileSystemStorage.java:331)
	at org.apache.falcon.entity.FileSystemStorage.evict(FileSystemStorage.java:300)
	... 23 more


Re: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem (JA018)

Rising Star

You can find logs by clicking on console url and clicking on the "1" under map tasks. I request you to share the hadoop version you are using.

Re: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem (JA018)

Explorer

it seems that the location data path must be like :

          <location type="data" path="/tmp/demo-falcon/process/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>

Re: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem (JA018)

From the Feed Specification page: A location tag specifies the type of location like data, meta, stats and the corresponding paths for them. A feed should at least define the location for type data, which specifies the HDFS path pattern where the feed is generated periodically. ex: type="data" path="/projects/TrafficHourly/${YEAR}-${MONTH}-${DAY}/traffic" The granularity of date pattern in the path should be at least that of a frequency of a feed.

So if your frequency is for example 1 hour, you need also ${HOUR} but no ${MINUTE}

Re: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem (JA018)

Explorer

What is the answer to this problem ??

Re: org.apache.falcon.FalconException: Couldn't evict feed from fileSystem (JA018)

Rising Star

Apart from defining location for data, can you please try setting "stats" and "meta" to "/tmp/${YEAR}-${MONTH}-${DAY}" ? Please let us know if this works. Also, please let me know the Java version you are using, I suspect the change in behavior is due to possible change in how pattern matcher behaves between Java 7 and 8.