i'm using QueryDatabaseTable processor to incrementally load data from a database table using two columns to perform the incremental load. However, I noticed that the query that fetches the fresh rows is as follows:
Select * from myTable where (mycolumn1 > someValue AND myclumn2 > somevalue)
However, this does not meet my business need and I would like to perform the following to fetch new records:
Select * from myTable where (mycolumn1 > someValue OR mycolumn2 > somevalue)
How can this be achieved?
Appreciate your help. Thanks
I would recommend creating a view with that query and have it watch some synthetic column.
Or in NiFi use two QueryDatabaseTableRecord processors.
And use QueryDatabaseTableRecord not QueryDatabaseTable.