Team: I have noticed one annoying behavior of falcon,Actually for a job in hdp2.2 falcon was retrying to run jobs 10 times but in new current version it is just failing after 1 try. So I just want to know whether its a change in hdp2.3 or some default behavior which we can change accordingly ? Thanks in advance. ... View more

@Team, I want to use Expression Language in Feed Entity, just like the case we do in process entity For Example: Process.xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <process name="yearWiseReplication-process" xmlns="uri:falcon:process:0.1"> <clusters> ..... <properties> <property name="feed_date" value="${formatTime(dateOffset(instanceTime(), -1, 'DAY'),'yyyy-MM-dd')}" /> </properties> .... </process> Wanted to use in the following away, But I'm failing Feed.xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <feed name="yearWiseReplication-output-feed" description="yearWiseReplication-output-feed" xmlns="uri:falcon:feed:0.1"> .... <locations> <location type="data" path="/hdptmp/availabilityYear/${YEAR}-${MONTH}-${DAY-1}" /> </locations> .... </feed> My objective is to run the my job which processes previous day data(yesterday's data) ... View more

Hi Team, I was reading a KB article which can help us to protect our HDFS dir, but when I tested it then I am able to delete a protected dir. Actually I have configured fs.protected.directories in core-site.xml with /lowes/sampleTest dir and tested below. [root@samplehost ~]$ hadoop fs -rm -R -skipTrash /lowes/sampleTest rm: Cannot delete non-empty protected directory /lowes/sampleTest [root@samplehost ~]$ hadoop fs -rm -R /lowes/sampleTest 16/04/27 05:50:15 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 360 minutes, Emptier interval = 0 minutes. Moved: 'hdfs://HDPINFHA/lowes/sampleTest' to trash at: hdfs://HDPINFHA/user/root/.Trash/Current So do you have any help on that. ... View more

Team: Can we change location of temp staging file .hive-staging_hive_2016-04-13_08-1* which is getting create inside /apps/hive/warehouse whenever we are running some create table as select statement. CREATE TABLE test_survey_answers_2016_01_04_oldrecords_temp AS SELECT AM.question_id AS question_id_old, AM.answer_id AS answer_id_old, AM.value AS value_old, AM.text AS text_old, AM.start_date AS start_date_old, AM.end_date as end_date_old, AM.load_date_time as load_date_time_old, AMD.question_id AS question_id_new, AMD.answer_id AS answer_id_new, AMD.value AS value_new, AMD.text AS text_new FROM test_survey_answers AM LEFT OUTER JOIN test_survey_answers_2016_01_04_daily_temp AMD ON (AM.question_id= AMD.question_id AND AM.answer_id=AMD.answer_id; I tried to change hive.metastore.warehouse.dir to some other dir but problem is new database also getting create in that location. So I want to have to diffrent dirs, one is for databases(i.e /apps/hive/warehouse) and another one is for temp staging data(like /tmp/ or something else). Can someone please help on this issue. ... View more