Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

QueryDatabaseTable processor to query rows that have been modified

QueryDatabaseTable processor to query rows that have been modified

New Contributor

Hi! i have used the QueryDatabaseTable processor to query rows incrementally, that is only the new rows added at the source after the last sync, but somehow i haven't been able to also get the rows that have been modified, is there an easy way to do this in nifi?

4 REPLIES 4

Re: QueryDatabaseTable processor to query rows that have been modified

Super Guru
@Luke Schnoller

If your table have some incremental columns that we can track of to pull incremental data i.e timestamp column (or) monotonically increasing id column etc.

Keep your incremental column in Max value column(s) property in QueryDatabaseTable processor.

Refer to this article for clear explanation of QueryDatabaseTable processor usage.

Re: QueryDatabaseTable processor to query rows that have been modified

New Contributor

Hi Shu! Thanks for the reply. I can already do an incremental fetch, what i'm asking for is a way to also get the rows that have been already synced, but that have been modified since last fetch. For this i would have a column with a last_modified date.

Re: QueryDatabaseTable processor to query rows that have been modified

New Contributor

Hi Luke Schnoller ,


I am also looing for same.I am able to get incremental data but what if rows is modified ,querydatabasetable processor is unable to fetch those modified records.Please let me know if u get this solution.

Re: QueryDatabaseTable processor to query rows that have been modified

New Contributor

I think that QueryDatabaseTable processor doesn't support fetching updated data, you can use CaptureDataChange Processor instead!