Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Nifi QueryDatabaseTable

avatar
New Contributor

Hello, 

I´m using QueryDatabaseTable with time_stamp max value column to get in real time +2k records per day. Despite I read all docs that I found I can not understand how it works. 

It supose to do a select every time a new record is inserted in the table but if I take a look in the postgres log I can see that the query is executed every second in search for new records.

I though that this processors works like a trigger but it seems like it doesn´t and I´m afraid that this can affect database performance.

Can somebody explain how this processor internally works?  

Thank you in advance

 

 

2 ACCEPTED SOLUTIONS

avatar
Super Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
4 REPLIES 4

avatar
Community Manager

@MihaiMaranduca, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @SAMSAL@cotopaul @MattWho @steven-matison  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Super Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
New Contributor

Good morning @cotopaul @SAMSAL First of all thank you very much for your time

Ok Ok so I see that I missunderstood how QueryDatabaseTable is "triggered". @SAMSAL I knew about those configuration but as I said I thought this processor only do the query when a new record is inserted in the database taking reference the maximum column value. Makes sense now the postgresql log...the querys executed every second since my RunSchedule is on default (0). 

What I use minifi for is to load sales from a supermarket then send it via API (and others transformations), so I need this to be in real time as soon as the client paid. I guess I have to define Run Schedule every few seconds. @cotopaul I will take a look at my DBCPConnectionPool, this info it´s helpfull

Thank you all.