Member since 
    
	
		
		
		12-03-2017
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                156
            
            
                Posts
            
        
                26
            
            
                Kudos Received
            
        
                11
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 2120 | 11-03-2023 12:17 AM | |
| 4195 | 12-12-2022 09:16 PM | |
| 1588 | 07-14-2022 03:25 AM | |
| 2427 | 07-28-2021 04:42 AM | |
| 3416 | 06-23-2020 10:08 PM | 
			
    
	
		
		
		03-26-2024
	
		
		04:23 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Hello Exeprts, 
 I am using hive-jdbc jar in my application to connect Hive and Delta (via Thrift server)  from Nifi flow using a custom processor - 
 https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc 
 Now the problem is this has not released any major version in recent times and all the available older versions have velnurables in it.  (Even I have tried '4.0.0-beta-1' , I don't see any velnurables with this, but looks like they have removed some internal dependencies, so it did not work for me) 
 So to get rid of velnurbles I am checking to see if there is any alternate option for me to use. 
 Thanks 
 Mahendra 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Hive
- 
						
							
		
			Apache NiFi
			
    
	
		
		
		03-12-2024
	
		
		12:20 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hello Experts,  I have a custom processor where I have below lines of code -   private ReadContext configJsonCtx = JsonPath.parse(configJsonString);      private List<Object> workFlows = configJsonCtx.read("$.Workflow");         this.workFlows.forEach((workFlowObj) -> {      try {      DocumentContext workFlow = JsonPath.parse(workFlowObj);     ........        This is working fine with Nifi 16 and com.jayway.jsonpath.json-path:2.4.0 dependency.    But now I am trying to upgrade to nifi 24 and I am getting below error when I do that  Any idea what could be the reason and solution for the same.     2024-03-12 05:09:08,815 ERROR [Timer-Driven Process Thread-320] c.o.n.p.o.O9QueueRouting O9QueueRouting[id=8718e632-f033-372b-8f6c-682399fbf9b6] Exception: java.lang.ClassCastException: class java.util.LinkedHashMap cannot be cast to class com.jayway.jsonpath.DocumentContext (java.util.LinkedHashMap is in module java.base of loader 'bootstrap'; com.jayway.jsonpath.DocumentContext is in unnamed module of loader org.apache.nifi.nar.NarClassLoader @147e0734)  java.lang.ClassCastException: class java.util.LinkedHashMap cannot be cast to class com.jayway.jsonpath.DocumentContext (java.util.LinkedHashMap is in module java.base of loader 'bootstrap'; com.jayway.jsonpath.DocumentContext is in unnamed module of loader org.apache.nifi.nar.NarClassLoader @147e0734)  at com.o9solutions.nifi.processors.o9_custom_processors.O9QueueRouting.lambda$onScheduled$0(O9QueueRouting.java:299)  at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)  at com.o9solutions.nifi.processors.o9_custom_processors.O9QueueRouting.onScheduled(O9QueueRouting.java:269)  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  at java.base/java.lang.reflect.Method.invoke(Method.java:566)  at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)  at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)  at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)  at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)  at org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$8(StandardProcessorNode.java:1765)  at org.apache.nifi.engine.FlowEngine$3.call(FlowEngine.java:123)  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)  at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)  at java.base/java.lang.Thread.run(Thread.java:829)      Thanks in advnace  Mahendra 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache NiFi
			
    
	
		
		
		03-08-2024
	
		
		02:20 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Thank you so much @MattWho ! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-05-2024
	
		
		08:46 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @MattWho - requesting your opinion here (if possible) 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-04-2024
	
		
		10:35 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Hello experts,  We are using Nifi for data ingestion via http - using 'HandleHttpRequest' processor.  Data is getting ingested continuously but once in a while when incoming volume is little high we get 'Service Unavailable - 503' error from nifi end point.  Below is the log we see in nifi app logs -    2024-03-04 11:20:25,558 WARN [StandardHttpContextMap-c2ffb1fd-2c09-3a90-0e5e-18384499605f] o.a.nifi.http.StandardHttpContextMap StandardHttpContextMap[id=c2ffb1fd-2c09-3a90-0e5e-18384499605f] Request from 10.13.2.204 timed out; responding with SERVICE_UNAVAILABLE     Imp configs :  [HandleHttpRequest processor]  Container Queue Size : 2000     [StandardHttpContextMap controller service]  Maximum Outstanding Requests : 8000  Request Expiration : 3min     nifi.web.jetty.threads=200 (in nifi.properties)  Maximum Timer Driven Thread Count = 200     Total number of running processors on this env : 4200+    This nifi is running on 8 core machine (ec2 instance)   When we look at the CPU usage during the issue we dont see a cpu usage is more than 40%         Can anyone help me to understand the issue and the fix?     Thanks  Mahendra 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache NiFi
			
    
	
		
		
		02-22-2024
	
		
		06:55 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thank you very much for the quick reply @cotopaul @MattWho .     2024-02-22 14:44:24,319 INFO [main] o.apache.nifi.registry.bootstrap.Command Launched Apache NiFi Registry with Process ID 4290  2024-02-22 14:44:25,086 INFO [NiFi Registry Bootstrap Command Listener] o.a.n.registry.bootstrap.RunNiFiRegistry Apache NiFi Registry now running and listening for Bootstrap requests on port 44846  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut Apache NiFi _ _  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut _ __ ___ __ _(_)___| |_ _ __ _ _  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut | '__/ _ \/ _` | / __| __| '__| | | |  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut | | | __/ (_| | \__ \ |_| | | |_| |  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut |_| \___|\__, |_|___/\__|_| \__, |  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut ==========|___/================|___/=  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut v  2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut  2024-02-22 14:44:31,494 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr Server start failed: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'accessPolicyResource' defined in file [/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/api/AccessPolicyResource.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'standardServiceFacade' defined in file [/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/service/StandardServiceFacade.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'registryService' defined in URL [jar:file:/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/lib/nifi-registry-framework-1.24.0.jar!/org/apache/nifi/registry/service/RegistryService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'databaseMetadataService' defined in URL [jar:file:/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/lib/nifi-registry-framework-1.24.0.jar!/org/apache/nifi/registry/db/DatabaseMetadataService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.license.FlywayEditionUpgradeRequiredException: Flyway Teams Edition or MySQL upgrade required: MySQL 5.7 is no longer supported by Flyway Community Edition, but still supported by Flyway Teams Edition.  2024-02-22 14:44:32,331 INFO [main] o.a.n.registry.bootstrap.RunNiFiRegistry NiFi Registry never started. Will not restart NiFi Registry  ^C  root@nifi12-dev:/usr/independent_nifi/nifi-registry-1.24.0/logs#  root@nifi12-dev:/usr/independent_nifi/nifi-registry-1.24.0/logs# mysql --version  mysql Ver 14.14 Distrib 5.7.33, for Linux (x86_64) using EditLine wrapper  root@nifi12-dev:/usr/independent_nifi/nifi-registry-1.24.0/logs#     I see above logs in bootstrap, Is the issue because my mysql version is 5.7?    In another server I am running nifi 1.16 registry with this mysql 5.7 version and its working there from 2 years.     Thanks  Mahendra 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-21-2024
	
		
		09:23 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hello experts,  I have downloaded and trying to run nifi registry 1.24 with meta data db as mysql and Git flow persistence.    Getting below error and unable to start the service -       log in text :   2024-02-21 16:11:13,716 INFO [main] o.a.n.registry.bootstrap.RunNiFiRegistry Runtime Java version: 11.0.14  2024-02-21 16:11:13,725 INFO [main] o.apache.nifi.registry.bootstrap.Command Starting Apache NiFi Registry...  2024-02-21 16:11:13,725 INFO [main] o.apache.nifi.registry.bootstrap.Command Working Directory: /usr/independent_nifi/nifi-registry-1.24.0  2024-02-21 16:11:13,726 INFO [main] o.apache.nifi.registry.bootstrap.Command Command: java -classpath /usr/independent_ni.............................  2024-02-21 16:11:13,743 INFO [main] o.apache.nifi.registry.bootstrap.Command Launched Apache NiFi Registry with Process ID 19099  2024-02-21 16:11:14,447 INFO [NiFi Registry Bootstrap Command Listener] o.a.n.registry.bootstrap.RunNiFiRegistry Apache NiFi Registry now running and listening for Bootstrap requests on port 37730  2024-02-21 16:11:17,712 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut  2024-02-21 16:11:17,712 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut Apache NiFi _ _  2024-02-21 16:11:17,712 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut _ __ ___ __ _(_)___| |_ _ __ _ _  2024-02-21 16:11:17,712 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut | '__/ _ \/ _` | / __| __| '__| | | |  2024-02-21 16:11:17,712 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut | | | __/ (_| | \__ \ |_| | | |_| |  2024-02-21 16:11:17,713 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut |_| \___|\__, |_|___/\__|_| \__, |  2024-02-21 16:11:17,713 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut ==========|___/================|___/=  2024-02-21 16:11:17,713 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut v  2024-02-21 16:11:17,713 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut  2024-02-21 16:11:20,550 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr Wed Feb 21 16:11:20 UTC 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.  2024-02-21 16:11:21,763 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr Server start failed: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'accessPolicyResource' defined in file [/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/api/AccessPolicyResource.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'standardServiceFacade' defined in file [/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/service/StandardServiceFacade.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'registryService' defined in URL [jar:file:/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/lib/nifi-registry-framework-1.24.0.jar!/org/apache/nifi/registry/service/RegistryService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'databaseMetadataService' defined in URL [jar:file:/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/lib/nifi-registry-framework-1.24.0.jar!/org/apache/nifi/registry/db/DatabaseMetadataService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flyway' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.flywaydb.core.Flyway]: Factory method 'flyway' threw exception; nested exception is org.flywaydb.core.api.FlywayException: Unable to obtain connection from Flyway DataSource  2024-02-21 16:11:22,745 INFO [main] o.a.n.registry.bootstrap.RunNiFiRegistry NiFi Registry never started. Will not restart NiFi Registry     nifi-registry.properties:    nifi.registry.web.war.directory=./lib  nifi.registry.web.http.host=nifi12-dev.xyz.com  nifi.registry.web.http.port=61080  nifi.registry.web.https.host=  nifi.registry.web.https.port=  nifi.registry.web.https.application.protocols=http/1.1  nifi.registry.web.jetty.working.directory=./work/jetty  nifi.registry.web.jetty.threads=200  nifi.registry.web.should.send.server.version=true  nifi.registry.security.keystore=  nifi.registry.security.keystoreType=jks  nifi.registry.security.keystorePasswd=  nifi.registry.security.keyPasswd=  nifi.registry.security.truststore=  nifi.registry.security.truststoreType=jks  nifi.registry.security.truststorePasswd=  nifi.registry.security.needClientAuth=True  nifi.registry.security.authorizers.configuration.file=./conf/authorizers.xml  nifi.registry.security.authorizer=managed-authorizer  nifi.registry.security.identity.providers.configuration.file=./conf/identity-providers.xml  nifi.registry.security.identity.provider=  nifi.registry.sensitive.props.additional.keys=  nifi.registry.providers.configuration.file=./conf/providers.xml  nifi.registry.registry.alias.configuration.file=./conf/registry-aliases.xml  nifi.registry.extensions.working.directory=./work/extensions  nifi.registry.db.directory=  nifi.registry.db.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE;useSSL=false  # database properties  nifi.registry.db.url=jdbc:mysql://localhost:3306/nifi24_registry  nifi.registry.db.driver.class=com.mysql.jdbc.Driver  nifi.registry.db.driver.directory=/usr/share/java/  nifi.registry.db.username=registry  nifi.registry.db.password=PasswordSample1234  nifi.registry.db.maxConnections=5  nifi.registry.db.sql.debug=false      Any suggestion / solution is much appreciated.  Thanks  Mahendra 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache NiFi
- 
						
							
		
			NiFi Registry
			
    
	
		
		
		02-05-2024
	
		
		07:13 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thanks for the quick reply @MattWho .  We found the issue.    It was due to firewall setup for the partition-2 on broker side.  When they fixed it started working 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-05-2024
	
		
		05:13 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Hello experts,  We are facing an issue, where preprod Nifi server is able to read a kafka topic correctly.  It is able to read all 3 partitions data.  At the same time UAT Nifi server is connecting to same topic with different consumer group id and which is reading only the partition 0 & partition 1 but not partition 2.  Screen shot from Lenses abt the same consumer group which is facing issue         Any idea what could be the possible reasons?  Thanks  Mahendra 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache NiFi
			
    
	
		
		
		11-10-2023
	
		
		02:55 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hello Experts,  We are using Nifi for continuous data ingestion over http port (as rest api) and also we use nifi admin rest api to dynamically start/stop procesors or update variable registry or create process groups etc.    We are facing a challenge with these admin rest api's (/nifi-api), i.e   when nifi restarted all apis responds very quickly and works fine and when days go on (if no nifi restart in btwn) they nifi rest api's becomes very slow and at some point they stop responding or timing out.  And this slowness/not responding issue will resolve only after nifi restart.  Since we have real time data ingestion via http port , we cant really restart frequently.  Nifi version : 1.16.3  Is there any solution for this problem ? or any suggestion to tackle this would be much appreciated  Thanks,  Mahendra 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache NiFi
 
        













