Member since 
    
	
		
		
		07-25-2022
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                29
            
            
                Posts
            
        
                7
            
            
                Kudos Received
            
        
                4
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 2068 | 11-06-2024 02:11 PM | |
| 1373 | 08-14-2024 01:31 PM | |
| 3078 | 08-14-2024 12:52 PM | |
| 1359 | 04-04-2024 08:45 AM | 
			
    
	
		
		
		06-06-2025
	
		
		12:01 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thanks @MattWho  for your valuable inputs. This is very helpful for conducting the correct analysis on our end. I will check the possibility of customizing the processors that connect to the database.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-06-2025
	
		
		09:21 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thank you @MattWho , for the detailed information. Could you please advise whether customization would be beneficial in our situation? Specifically, I would like to know if we can create a CustomDBConnectionService to establish a database connection in NiFi without any pooling. There is a centralized connection pool managed by PgBouncer in our environment that handles pooling.  Can you please clarify the below regarding DB connection customization.    Is it possible to create a custom database service without utilizing connection pooling?    If yes, what would be the impact on the performance of Apache NiFi when local pooling is not enabled?    However, I would like to ascertain whether the settings below would effectively lead to the soft or virtual disabling of connection pooling.        Property Name  Value  Comments    Max Idle Connections  0  Set to zero to make sure no idle connections in the pool    Minimum Idle Connections  0  Set to zero to make sure no idle connections in the pool    Minimum Evictable Idle Time  1 secs  Any idle connection will be eligible for eviction after 1 second    Soft Minimum Evictable Idle Time  1 secs  Any idle connection will be eligible for eviction after 1 second    Time Between Eviction Runs  1 secs  To run evictor thread every second    Max Connection Lifetime  -1  Can you clarify what does -1 indicate?  Does it mean no limit on the lifetime of a connection?    Max Total Connections  -1  Set to negative value to allow unlimited active connections    Max Wait Time  -1  Set to negative value to allow waiting indefinitely for connections    Validation query  Null  No validation of connection     Thanking you for your help!    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-02-2025
	
		
		09:56 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi,  All applications within our portfolio utilize PgBouncer as the database connection pool mechanism, and consequently, any application-level pooling is to be disabled. Apache NiFi is one of the applications required to adhere to this standard. We are seeking guidance on the options available to disable database connection pooling in NiFi. Could you please advise on how we can accomplish this? Our current nifi version is 2.1  Thanks for your help and let me know if more details are required.    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache NiFi
 
			
    
	
		
		
		04-30-2025
	
		
		10:27 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thank you @MattWho  for providing the details! I now have a much clearer understanding of what was happening.  I don't have detailed logs for the last occurrence, but I'll share them if we encounter the issue again. One thing is certain: the node was disconnected due to a discrepancy in the flow.json file. This happened during the deployment of NiFi flows, specifically while upgrading the version of an existing flow.   If it had been a standalone execution of the disconnected node, there wouldn't have been an issue, I believe. The actual problem arose because some flow files were stuck in one of the flows, at a connection where load balancing is enabled with the "partition by attribute" strategy (as shown in the image). I assume those records were waiting to be transferred to the other two nodes, which were not accessible to the disconnected node.   Could you explain this situation further for my better understanding? That will help us take the right steps. Please let me know if more details are required.              
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		04-25-2025
	
		
		02:53 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi  We are currently operating a three-node cluster using Apache NiFi version 2.1.  During a recent deployment of NiFi flows, we encountered an issue related to replication inconsistencies across the nodes.  This problem resulted in one of the nodes being out of the cluster and remained in disconnected state.  Furthermore, the affected node did not attempt to reconnect to the cluster on its own.  Although the node was disconnected from the cluster, it continued to operate independently, allowing it to consume messages, execute database queries, and perform actions as a standalone instance.  I would like to know two things here   Is this expected behavior for nifi node to be active although it is not in cluster  Can we alter the behavior through configuration to stop processors temporarily while node is not connected to cluster?   Thanks for in advance for your inputs.  Let me know if more details are required.  Thanks             
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache NiFi
 
			
    
	
		
		
		03-27-2025
	
		
		03:22 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thanks @Shelton for details!!  I will try Option 3 in our pipeline shell script.   Will let you know if any further issues. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-28-2025
	
		
		02:03 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thank you @MattWho for details.  As you mentioned, I will post my usecase in the jira.  Thanks for your help! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-28-2025
	
		
		08:52 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @MattWho       Thanks for info about PutDatabaseRecord cache. We would like to keep PutDatabaseRecord Table schema cache to reduce DB hits.   Is there any option in Nifi to force refresh this Table Schema cache?     
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-27-2025
	
		
		01:31 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi  My one liner question:  How can we ensure that the DatabaseTableSchemaRegistry recognizes changes in the table structure and updates the corresponding table schema definition?  Details:  Our PutDatabaseRecord processors' readers are configured to use the DatabaseTableSchemaRegistry to parse incoming JSON. We have observed multiple instances where the NiFi flow fails when the target table structure is altered, such as when columns are added or removed.  When new columns are added to the target table, the PutDatabaseRecord reader does not recognize the change and continues to create insert statements without the new columns. We are unsure what may have resolved the issue, but in some instances, it was automatically resolved after a day. We suspect there may be an internal cache that is rebuilt periodically. We have no clear solution for this issue and would greatly appreciate any assistance regarding it.  Our NiFi system is a 3-node cluster (version 2.1) that uses Kubernetes-based leader election and state management, without an external ZooKeeper.  Thanks          
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache NiFi
 
			
    
	
		
		
		11-06-2024
	
		
		02:23 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I was able to setup GitLabFlowRegistry in controller settings successfully (which is new feature in Nifi 2.0)  From Nifi UI, I am able to import Nifi flow directly from Gitlab repository.  After modifications, I was able to successfully commit into Gitlab repo, from Nifi UI.    Do we have any toolkit commands to perform these operations?  Currently we are using toolkit commands to deploy Nifi flow changes via CICD pipeline.  Can I achieve the same with new GitLabFlowRegistry?    Appreciate any help/pointers.     Thanks       
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache NiFi