and please suggest how i can set batch_ts value in Update record processor . ... View more

Hi @Matt Burgess, We are using PutHDFS processor and further we have use Replace text processor to execute load data in path commond . I can see we have UpdateRecord processor so as per your suggestion i can use this processor after convertAvroto Orc processor ?. below is the flow detail . Regards, Shantanu ... View more

Hello @William Brooks i tired your options but looks like it not working.It has executed on the same time and 2 flow file inserted in to down stream . ... View more

Hello @Shu .I think you are giving me a reply based on subject line but now problem is as below. In above screen shot you can see the setting for Monitor activity with this setting my jobs are running in productions everyday but problem occurred when Querydatabase table processor triggered in different instance and monitor activity processor triggered on different instance and in this case since data is there so at 9 am Querydatabase table processor produce one flow file and its loaded to table and now at 10 Am monitor activity also gets executed and its generate one flow file and again my table got loaded with 0 count data .Can we restrict both processor can run on same instance or any other solution you have for this issue. Monitor activity is generating flow file in both case when querydatabses table processor is generatng flow file in cases of delta data and if querydatabses table processor is not generatngflow file in cases of delta data. We need flow file in case of when querydatabses table processor is not generatngflow file . Regards, Shantanu ... View more

Thanks you @William Brooks I will apply your concept and let you know the result. Thanks, Shantanu ... View more

Hello @Shu I am facing some issue with Monitor activity . If you do not mind can you help me on it. In above screen shot you can see the setting for Monitor activity with this setting my jobs are running in productions everyday but problem occurred when Querydatabase table processor triggered in different instance and monitor activity processor triggered on different instance and in this case since data is there so at 9 am Querydatabase table processor produce one flow file and its loaded to table and now at 10 Am monitor activity also gets executed and its generate one flow file and again my table got loaded with 0 count data .Can we restrict both processor can run on same instance or any other solution you have for this issue. Regards, Shantanu. ... View more

here is the details. ... View more

Thank you We are already using Monitoring Scope = cluster but still the problem is there. ... View more

Thanks for the reply @shu and @Dukool SHarma . I am not writing any Map and reduce job. On the query level i.e in hive query can we call this LazyOutputFormat?. My table formate is ORC. Regards, Shantanu ... View more

Can you please explain how we can implement this concept? Regards, Shantanu. ... View more

If i will set Continually Send Messages is true then everyday if data will not available then i can get flow file? right now it has in false stat. ... View more

Another thing wanted to add here .For copy Attributes we need to set True i have given false .You can look in to below screen shot. Regards, Shantanu ... View more

Thanks Shu, I have used above Cron scheduled but today the behaviour of Monitor activity is very strange, yesterday it ran successfully when in QueryData base processor was not getting any delta record it has generated flow file and sent to execute sql but today again there was not data available for query databases but today Monitor activity not triggered. I do not know what I am missing here flow is same as above just in scheduling i have given 0 0-1 10 1/1 * ? * with threshold value 1 min. Could you please suggest here .I am attaching flow setting for Monitory activity. I have given Execution on Promary node is there any thing with this and Automatically Terminate Relationships i have given activity.restored and success while connecting to QueryDatabases and inactive while connecting to Execute Sql. Please have a look in below screen shot for more detail Regards, Shantanu. ... View more

Hello Shu. Sorry to bother you again.just I am curious for above Cron schedule 0 0-1 10 1/1 * ? *,will work on daily base or only for one day because my jobs scheduled to run daily.could you please give me a reply on it Regards Shantanu ... View more

Thanks Shu. what is 1/1 in your corn scheduling ? will it impact for next run if i will schedule on 0 0-1 10 1/1 * ? *, Regards, Shnatanu ... View more

Thank you Shu, If i will change the threshold value 5 min to 1 min then in this cases i need to change scheduling of Monitor Activity processor? Regards, Shantanu ... View more

Thank you for your quick response . Here in my scenario lettle different I have a .txt file with delimited '|' and we have some 10 columns and one of the column is STORE_ID So i wanted to split that file in to 2 part based on STORE ID value .ex: STORE ID 1 to 10 will go to different and 10 to 20 will go to different file.In your above solution how can i achieve this value range ? Please suggest me . Regards, Shantanu ... View more

@shu Can we do this by using UpdateAttribute processor with Advance options if possilbe please explain it. ... View more