Member since
12-03-2017
138
Posts
18
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
565 | 11-03-2023 12:17 AM | |
2044 | 12-12-2022 09:16 PM | |
732 | 07-14-2022 03:25 AM | |
1228 | 07-28-2021 04:42 AM | |
1568 | 06-23-2020 10:08 PM |
04-23-2024
01:46 AM
1 Kudo
Not possible, there is a reason for that if we allow to add a property to set an initial offset, every time that processor was stopped and started we would again start at that configured offset, with console consumer, you only pass the initial offset on the first execution then subsequent executions use the last stored offset in Kafka, which is not the case here with the processor. Thus only two options are "earliest" or "latest". Thank you
... View more
03-28-2024
09:51 AM
1 Kudo
@hegdemahendra You may try Cloudera Hive JDBC Driver. The driver class name would be "com.cloudera.hive.jdbc.HS2Driver".
... View more
03-26-2024
07:33 AM
hi @hegde , thank you for your fast reply. This would have worked indeed. Fortunately, we found the culprit on our matter. Someone made manual changes to the nifi github repo which was used by the nifi-registry. We reverted the changes in github and now the registry seems to work again 🙂
... View more
03-12-2024
12:20 AM
Hello Experts, I have a custom processor where I have below lines of code - private ReadContext configJsonCtx = JsonPath.parse(configJsonString); private List<Object> workFlows = configJsonCtx.read("$.Workflow"); this.workFlows.forEach((workFlowObj) -> { try { DocumentContext workFlow = JsonPath.parse(workFlowObj); ........ This is working fine with Nifi 16 and com.jayway.jsonpath.json-path:2.4.0 dependency. But now I am trying to upgrade to nifi 24 and I am getting below error when I do that Any idea what could be the reason and solution for the same. 2024-03-12 05:09:08,815 ERROR [Timer-Driven Process Thread-320] c.o.n.p.o.O9QueueRouting O9QueueRouting[id=8718e632-f033-372b-8f6c-682399fbf9b6] Exception: java.lang.ClassCastException: class java.util.LinkedHashMap cannot be cast to class com.jayway.jsonpath.DocumentContext (java.util.LinkedHashMap is in module java.base of loader 'bootstrap'; com.jayway.jsonpath.DocumentContext is in unnamed module of loader org.apache.nifi.nar.NarClassLoader @147e0734) java.lang.ClassCastException: class java.util.LinkedHashMap cannot be cast to class com.jayway.jsonpath.DocumentContext (java.util.LinkedHashMap is in module java.base of loader 'bootstrap'; com.jayway.jsonpath.DocumentContext is in unnamed module of loader org.apache.nifi.nar.NarClassLoader @147e0734) at com.o9solutions.nifi.processors.o9_custom_processors.O9QueueRouting.lambda$onScheduled$0(O9QueueRouting.java:299) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at com.o9solutions.nifi.processors.o9_custom_processors.O9QueueRouting.onScheduled(O9QueueRouting.java:269) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55) at org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$8(StandardProcessorNode.java:1765) at org.apache.nifi.engine.FlowEngine$3.call(FlowEngine.java:123) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Thanks in advnace Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
03-08-2024
02:20 AM
1 Kudo
Thank you so much @MattWho !
... View more
03-06-2024
02:44 AM
2 Kudos
Hello @hegdemahendra, 1) Please refer to the following article to connect Kafka from Nifi: https://community.cloudera.com/t5/Community-Articles/Integrating-Apache-NiFi-and-Apache-Kafka/ta-p/247433 2) Also, to isolate the issue you can try to connect Kafka from the same settings from nifi node using the Kafka command Please let us know if you still have any questions regarding the same or facing any issues. We will be happy to assist you with it. If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post. Thank you.
... View more
02-22-2024
06:55 AM
Thank you very much for the quick reply @cotopaul @MattWho . 2024-02-22 14:44:24,319 INFO [main] o.apache.nifi.registry.bootstrap.Command Launched Apache NiFi Registry with Process ID 4290 2024-02-22 14:44:25,086 INFO [NiFi Registry Bootstrap Command Listener] o.a.n.registry.bootstrap.RunNiFiRegistry Apache NiFi Registry now running and listening for Bootstrap requests on port 44846 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut Apache NiFi _ _ 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut _ __ ___ __ _(_)___| |_ _ __ _ _ 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut | '__/ _ \/ _` | / __| __| '__| | | | 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut | | | __/ (_| | \__ \ |_| | | |_| | 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut |_| \___|\__, |_|___/\__|_| \__, | 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut ==========|___/================|___/= 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut v 2024-02-22 14:44:28,308 INFO [NiFi logging handler] org.apache.nifi.registry.StdOut 2024-02-22 14:44:31,494 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr Server start failed: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'accessPolicyResource' defined in file [/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/api/AccessPolicyResource.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'standardServiceFacade' defined in file [/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/service/StandardServiceFacade.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'registryService' defined in URL [jar:file:/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/lib/nifi-registry-framework-1.24.0.jar!/org/apache/nifi/registry/service/RegistryService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'databaseMetadataService' defined in URL [jar:file:/usr/independent_nifi/nifi-registry-1.24.0/work/jetty/nifi-registry-web-api-1.24.0.war/webapp/WEB-INF/lib/nifi-registry-framework-1.24.0.jar!/org/apache/nifi/registry/db/DatabaseMetadataService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.license.FlywayEditionUpgradeRequiredException: Flyway Teams Edition or MySQL upgrade required: MySQL 5.7 is no longer supported by Flyway Community Edition, but still supported by Flyway Teams Edition. 2024-02-22 14:44:32,331 INFO [main] o.a.n.registry.bootstrap.RunNiFiRegistry NiFi Registry never started. Will not restart NiFi Registry ^C root@nifi12-dev:/usr/independent_nifi/nifi-registry-1.24.0/logs# root@nifi12-dev:/usr/independent_nifi/nifi-registry-1.24.0/logs# mysql --version mysql Ver 14.14 Distrib 5.7.33, for Linux (x86_64) using EditLine wrapper root@nifi12-dev:/usr/independent_nifi/nifi-registry-1.24.0/logs# I see above logs in bootstrap, Is the issue because my mysql version is 5.7? In another server I am running nifi 1.16 registry with this mysql 5.7 version and its working there from 2 years. Thanks Mahendra
... View more
02-05-2024
07:13 AM
Thanks for the quick reply @MattWho . We found the issue. It was due to firewall setup for the partition-2 on broker side. When they fixed it started working
... View more
12-01-2023
07:13 AM
@hegdemahendra You also need to be careful with some rest-api request as they may generate reports that are held in NiFi heap until a rest-api call is made to remove them. Doing similar actions directly from the UI handles the multiple calls needed automatically. So calls like these can slowly eat away at the NiFi heap impacting performance until a restart. Provenance queries would be an example of this. Unrelated note: NiFi Variable registry has been deprecated in favor of NiFi Parameters now. The Variable registry functionality has been officially removed starting with the latest NiFi 2.0.0-M1 release. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
11-03-2023
12:17 AM
1 Kudo
Found the issue, DisrtibtedMapCacheServer was configured with default 1 mb to handle in each cache request. But I was trying to pass a attribute value to pass to wait processor from Notify which was 1.5 mb. After increasing the cache server config from 1mb to 10 mb it worked
... View more