Dynamic Initial Max Value on GenerateTableFetch

[#1]

In a GenerateTableFetch i have set as Maximum-value Columns the column updated_at. In addition i have define the property "initial.maxvalue.updated_at" with an hard-code initial value. However i do not want to hardcode manual the initial value but instead grab it dynamically, like performing an SQL statement to get the max updated value from the table. How i could feed such information in "initial.maxvalue.updated_at" ?

[#2 · Accepted solution]

Hi @TonyPath ,

I noticed that the added dynamic property initial.maxvalue.<max_column_name> accept expression language (EL) so since the GenerateTableFetch can take incoming relationship maybe you can use  ExecuteSQLRecord to fetch the initial max value from sql into json format  then store it in a flow file attribute using EvaluateJsonPath processor which you can reference to set  the initial.maxvalue.<max_column_name> property value as follows:

ExecuteSQLRecord Config:

EvaluateJsonPath Config:

GenerateTableFetch Config:

If that helps please accept solution.

Thanks

[#3]

Hi @SAMSAL, I haven't tried it yet but your proposal makes sense, but the execute sql record will run constantly and required manual intervention to stop it so to produce only one flow file. However this is another case

[#4]

How often are you looking to run the GenerateFetchTable ? If its going to be a batch process then you can set up the a schedule on the top processor using processor config -> Scheduling tab and setting the Run Schedule value. By default this value is set to 0 secs which means its continuously running.