Created 07-31-2023 01:36 AM
Hello,
I´m using QueryDatabaseTable with time_stamp max value column to get in real time +2k records per day. Despite I read all docs that I found I can not understand how it works.
It supose to do a select every time a new record is inserted in the table but if I take a look in the postgres log I can see that the query is executed every second in search for new records.
I though that this processors works like a trigger but it seems like it doesn´t and I´m afraid that this can affect database performance.
Can somebody explain how this processor internally works?
Thank you in advance
Created 07-31-2023 07:09 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 07-31-2023 11:59 PM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 07-31-2023 02:34 AM
@MihaiMaranduca, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @SAMSAL, @cotopaul @MattWho @steven-matison who may be able to assist you further.
Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.
Regards,
Vidya Sargur,Created 07-31-2023 07:09 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 07-31-2023 11:59 PM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 08-01-2023 11:35 PM
Good morning @cotopaul @SAMSAL First of all thank you very much for your time
Ok Ok so I see that I missunderstood how QueryDatabaseTable is "triggered". @SAMSAL I knew about those configuration but as I said I thought this processor only do the query when a new record is inserted in the database taking reference the maximum column value. Makes sense now the postgresql log...the querys executed every second since my RunSchedule is on default (0).
What I use minifi for is to load sales from a supermarket then send it via API (and others transformations), so I need this to be in real time as soon as the client paid. I guess I have to define Run Schedule every few seconds. @cotopaul I will take a look at my DBCPConnectionPool, this info it´s helpfull
Thank you all.