QueryDatabaseTable processor to query rows that have been modified

[#1]

Hi! i have used the QueryDatabaseTable processor to query rows incrementally, that is only the new rows added at the source after the last sync, but somehow i haven't been able to also get the rows that have been modified, is there an easy way to do this in nifi?

[#2]

@Luke Schnoller

If your table have some incremental columns that we can track of to pull incremental data i.e timestamp column (or) monotonically increasing id column etc.

Keep your incremental column in Max value column(s) property in QueryDatabaseTable processor.

Refer to this article for clear explanation of QueryDatabaseTable processor usage.

[#3]

Hi Shu! Thanks for the reply. I can already do an incremental fetch, what i'm asking for is a way to also get the rows that have been already synced, but that have been modified since last fetch. For this i would have a column with a last_modified date.

[#4]

Hi Luke Schnoller ,

I am also looing for same.I am able to get incremental data but what if rows is modified ,querydatabasetable processor is unable to fetch those modified records.Please let me know if u get this solution.

[#5]

I think that QueryDatabaseTable processor doesn't support fetching updated data, you can use CaptureDataChange Processor instead!