Member since
11-25-2021
12
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
14104 | 04-25-2022 11:05 PM |
05-30-2023
03:28 AM
Sorry @Neil_1992 where could I find nifi.service file, to add this property ?
... View more
02-28-2022
05:08 AM
1 Kudo
Few points when connecting to any DataBase from NiFi NiFi has Generic controller service named DBCPConnectionPool which used to create connection pool to given database and processors such as ExecuteSQL or PutSql will use this controller service to obtain connection and run the given queries. What is needed as minimum prerequisite to start with DBCPConnectionPool 1. Determine type of Database 2. Determine the JDBC URL syntax supported by Database 3. The Database client Driver which works outside of NiFi to connect to same Database needs to be placed on each NiFi nodes on given location which can be read user who is running NiFi as service All above needs to be provided by user , if Ucanaccess 5.0.1 client Driver works outside of NiFi then it should work with DBCPConnectionPool as well, Where you are getting this message "Given file does not exist" ? Is client driver is present on NiFi hosts and configured Database Driver Location is correct ? permissions? You mentioned There is a Cdata driver which works but that is a licensed product" , it works from where ? from NiFi:DBCPConnectionPool? Thank You.
... View more
01-11-2022
08:37 AM
@Neil_1992 I strongly recommend not setting your NiFi heap to 200GB. Java reserves the XMS space and grows to the XMX space as space is requested. Java Garbage Collection (GC) execution to reclaim heap no longer being used does not kick in to ~80% of heap is used. That means GC in your case would kick in at around 160+ GB of used heap. All GC execution is stop-the-world activity which means your NiFi will stop doing anything until GC completes. This can lead to long pauses resulting node disconnections, issues with timeouts with dataflows to external services ,etc. When it comes to flow.xml.gz file, you are correct that it is uncompressed and loaded into heap memory. The flow.xml.gz contains: Everything you add via the NiFi UI to the canvas (processors, RPG, input/output ports, funnels. labels, PGs, controller services, reporting tasks, connections, etc.). This includes all the configuration of fro each of those components. NiFi Templates are also stored in the flow.xml.gz, uncompressed and loaded in to heap as well. Once a NiFi template is created, it should be downloaded, stored outside of NiFi, and local copy of template inside of NiFi deleted. As far as your specific flow.xml.gz, I see a closing tag "</property>" following that indicates that some component has a property which typically consists of a "name" and "value" with the huge null laced null strings in the value field. I'd scroll up to see which component this property belongs to and then check why this value was set. Maybe it was a copy paste issue? Maybe this is just part of some template that was created with this large string for some purpose? Nothing here says with any certainty that there is a bug. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more