@MihaiMaranduca, In addition to what @SAMSAL already said, I would add the followings: - the NiFi processors WORK and DO NOT work as triggers. Let's remember that the triggers (in any sql database) are basically some database objects, whereas NiFi is a standalone application, connected to that Database. Using the Maximum-value Columns you will somehow mimic the trigger like functionality but in the end, NiFi will only execute an SQL Statement on your database, based on the column you have defined in the mentioned property. When it comes to performance optimization, I would personally recommend you the following: - Within QueryDatabaseTable, define the scheduling as per your requirements. If you know that your data gets loaded once per day, switch to an once per day execution .... if you need hourly, get an hourly execution and so on. - Within your DBCPConnectionPool, pay attention to the following properties, as they are pretty important: a) Time Between Eviction Runs = if you leave it default, most likely (depending on how the DB was configured as well) you will never close your sessions, eventually causing problems. Here you can set any value in millis, which will tell NiFi to drop the connection after that time has passed. For example, I am using 1440000 millis. b) Max Idle Connections = The maximum number of connection that can remain idle without extra ones being released. I personally am working with a lower value than the one default. c) Minimum Evictable Idle Time = The minimum time a connection may sit idle in the pool, before being marked for release. Here the value is 30 mins default but I am using 10 mins mostly, especially for the queries which have to be executed every 15mins.
... View more
