Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

what could be the reason behind failing Falcon Job?

what could be the reason behind failing Falcon Job?

Rising Star

Hello,

I have followed a below link for designing end-to-end pipeline.I am trying to process the data using pig script and store it some where on HDFS location.For that,I have created to Feed entities(raw-input-feed and filtered-feed) and one process entity as "pigProcess".

My HDFS Loaction for input Feed is:

/apps/falcon/primaryCluster/inputData/2015-05-16 and

Output Feed Location is

/apps/falcon/primaryCluster/outputData/2015-05-16

Here are my XML's:

1) raw-input-feed.xml

<feed xmlns='uri:falcon:feed:0.1' name='raw-input-feed' description='This is input Feed'> <groups>inputDataGroup</groups> <frequency>days(1)</frequency> <timezone>GMT+05:50</timezone> <clusters> <cluster name='primaryCluster' type='source'> <validity start='2016-05-17T05:25Z' end='2016-05-18T05:20Z'/> <retention limit='days(1)' action='delete'/> <locations> <location type='data'> </location> <location type='stats'> </location> <location type='meta'> </location> </locations> </cluster> </clusters> <locations> <location type='data' path='/apps/falcon/primaryCluster/inputData/${YEAR}-${MONTH}-${DAY}'> </location> <location type='stats' path='/'> </location> <location type='meta' path='/'> </location> </locations> <ACL owner='ambari-qa' group='users' permission='0755'/> <schema location='/none' provider='/none'/> <properties> <property name='jobPriority' value='NORMAL'> </property> </properties> </feed>

------------------------------------------------------------------------------

2) filtered-feed.xml

<feed xmlns='uri:falcon:feed:0.1' name='filtered-feed' description='This is Output Feed'> <groups>outputGroupData</groups> <frequency>days(1)</frequency> <timezone>GMT+05:50</timezone> <clusters> <cluster name='primaryCluster' type='source'> <validity start='2016-05-17T05:25Z' end='2016-05-18T05:20Z'/> <retention limit='days(1)' action='delete'/> <locations> <location type='data'> </location> <location type='stats'> </location> <location type='meta'> </location> </locations> </cluster> </clusters> <locations> <location type='data' path='/apps/falcon/primaryCluster/outputData/${YEAR}-${MONTH}-${DAY}'> </location> <location type='stats' path='/'> </location> <location type='meta' path='/'> </location> </locations> <ACL owner='ambari-qa' group='users' permission='0755'/> <schema location='/none' provider='/none'/> <properties> <property name='jobPriority' value='NORMAL'> </property> </properties> </feed>

-------------------------------------------------------------------------------------------------------

3) pigProcess.xml

<process xmlns='uri:falcon:process:0.1' name='pigProcess'> <clusters> <cluster name='primaryCluster'> <validity start='2016-05-17T12:10Z' end='2016-05-18T12:16Z'/> </cluster> </clusters> <parallel>1</parallel> <order>FIFO</order> <frequency>days(1)</frequency> <timezone>GMT+05:50</timezone> <inputs> <input name='input' feed='raw-input-feed' start='now(0,0)' end='now(0,0)'> </input> </inputs> <outputs> <output name='output' feed='filtered-feed' instance='now(0,0)'> </output> </outputs> <workflow name='myFlow' version='pig-0.13.0' engine='pig' path='/apps/falcon/primaryCluster/scripts/simple.pig'/> <retry policy='periodic' delay='minutes(30)' attempts='2'/> <ACL owner='ambari-qa' group='users' permission='0755'/> </process>

-------------------------------------------------------------------------------------------------

Any My pig Script is:

A = LOAD '$input' using PigStorage(',') AS (id:chararray, value:chararray, errcode:int); B = FILTER A BY (errcode == 200); STORE B INTO '$output' USING PigStorage('|');

The link(Followed to design same example):

http://hortonworks.com/blog/introduction-apache-falcon-hadoop/

but when I am scheduling all those entities I am getting error as

Failed Post-Processing-->

java.lang.NoClassDefFoundError: javax/jms/Message

and eviction error like

java.lang.NoClassDefFoundError: org/apache/commons/el/ExpressionEvaluatorImpl

1 REPLY 1

Re: what could be the reason behind failing Falcon Job?

Rising Star
@Manoj Dhake

Can you please confirm that oozie is configured correctly for Falcon to work? You can follow doc at https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_installing_manually_book/content/configu...