About MihaiMaranduca

MihaiMaranduca · ‎12-15-2023

Good morning. I have a minifi service in many servers that take data from them and insert it into a database located in another server. Yesterday I faced a problem with one of those servers. The switch died, and for 30 min there was no connection. All good this shouldn´t be a problem but it was... Minifi had 8 records in flowfile while problem started and it didn´t release them and catch the rest of the records until I restart the process mannualy. No error Log, no clue why this happenned...I´m not looking for a direct solution but for some advice or process I can look for. Thank you.

MihaiMaranduca · ‎08-01-2023

Good morning @cotopaul @SAMSAL First of all thank you very much for your time Ok Ok so I see that I missunderstood how QueryDatabaseTable is "triggered". @SAMSAL I knew about those configuration but as I said I thought this processor only do the query when a new record is inserted in the database taking reference the maximum column value. Makes sense now the postgresql log...the querys executed every second since my RunSchedule is on default (0). What I use minifi for is to load sales from a supermarket then send it via API (and others transformations), so I need this to be in real time as soon as the client paid. I guess I have to define Run Schedule every few seconds. @cotopaul I will take a look at my DBCPConnectionPool, this info it´s helpfull Thank you all.

MihaiMaranduca · ‎07-31-2023

Hello, I´m using QueryDatabaseTable with time_stamp max value column to get in real time +2k records per day. Despite I read all docs that I found I can not understand how it works. It supose to do a select every time a new record is inserted in the table but if I take a look in the postgres log I can see that the query is executed every second in search for new records. I though that this processors works like a trigger but it seems like it doesn´t and I´m afraid that this can affect database performance. Can somebody explain how this processor internally works? Thank you in advance

Online	Offline
Last Visited	‎12-15-2023 05:35 AM

Member Since	‎07-31-2023 12:53 AM
Last Visited	‎12-15-2023 05:35 AM
Posts	3

Cloudera Community

Minifi Connection Error

Re: Nifi QueryDatabaseTable

Nifi QueryDatabaseTable