Created 01-30-2019 10:10 PM
Hi! i have used the QueryDatabaseTable processor to query rows incrementally, that is only the new rows added at the source after the last sync, but somehow i haven't been able to also get the rows that have been modified, is there an easy way to do this in nifi?
Created 01-31-2019 01:09 AM
If your table have some incremental columns that we can track of to pull incremental data i.e timestamp column (or) monotonically increasing id column etc.
Keep your incremental column in Max value column(s) property in QueryDatabaseTable processor.
Refer to this article for clear explanation of QueryDatabaseTable processor usage.
Created 01-31-2019 12:42 PM
Hi Shu! Thanks for the reply. I can already do an incremental fetch, what i'm asking for is a way to also get the rows that have been already synced, but that have been modified since last fetch. For this i would have a column with a last_modified date.
Created 02-18-2019 10:44 AM
Hi Luke Schnoller ,
I am also looing for same.I am able to get incremental data but what if rows is modified ,querydatabasetable processor is unable to fetch those modified records.Please let me know if u get this solution.
Created 09-04-2019 07:15 AM
I think that QueryDatabaseTable processor doesn't support fetching updated data, you can use CaptureDataChange Processor instead!