Member since
02-01-2018
15
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5014 | 03-06-2018 06:14 PM |
04-12-2018
09:39 PM
I am trying to setup 2 separate NiFi clusters each with 3 nodes. I have setup the cluster protocol port different on both but using the same Zookeeper quorum. Seems like both clusters are trying to connect to the cluster coordinator of the first cluster. Is this a possibility, to use same Zookeeper quorum for 2 separate NiFi clusters? If not, what is the suggested way to do it?
... View more
Labels:
- Labels:
-
Apache NiFi
03-06-2018
07:19 PM
I got it working. I had to add the custom naming fields used in the Grok expression into the pattern file.
... View more
03-06-2018
06:14 PM
I got the pattern file created and used in Nifi. I am using this in grok expression: <(?<priority>[0-9]+)>(?<sequence>[0-9]+): *(\*)?%{CISCOTIMESTAMP}: (?<host>[a-zA-Z0-9_]+): %(?<facility>[A-Z0-9_]+)-(?<severity>[0-7]+)-(?<mnemonic>[A-Z0-9_]+): (?<message>.+) grok pattern file - https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/TestExtractGrok/patterns This expression works fine on Grok debugger site - http://grokdebug.herokuapp.com/. But not on Nifi. What am i doing wrong? Sample cisco router log data i am using: <189>22: *Apr 29 13:58:40.411: user: %SYS-5-CONFIG_I: Configured from console by console
... View more
03-05-2018
04:24 PM
@Pierre Villard - Thanks for the reply. I looked at this pattern file. But I am not sure how to link this file to Grok pattern file on nifi. I am running it on docker compose, so how do i store this pattern file and what path to provide in the Nifi?
... View more
03-02-2018
10:18 PM
I am running the Nifi on Docker. Nifi ParseSyslog fails for Cisco syslog, so trying to write custom regex parsing using Extract Grok processor. What is the Grok pattern file to be provided? I provided Grok expression, but it still looks for Grok pattern file. Any pointers on this will help. Thanks!
... View more
Labels:
- Labels:
-
Apache NiFi
02-26-2018
09:12 PM
@Alex Woolford Used option2 and set the Nifi to listen on that port. Works fine. Thanks!
... View more
02-13-2018
11:55 PM
@Alex Woolford I did the option 1 and the error is no longer there. But still the data is not getting loaded in the syslog loader. What could be wrong and is there any basic troubleshooting i can do?
... View more
02-13-2018
10:41 PM
We are trying to capture the syslog in Nifi and it shows the following error on ListenSyslog processor. Set up the UDP port 514 on the processor as well. Any pointers? In nifi-app.log, ERROR [StandardProcessScheduler Thread-3] o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1483) at org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:103) at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1302) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:206) at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1466) ... 9 common frames omitted Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1306) at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1302) ... 6 common frames omitted Caused by: java.net.SocketException: Permission denied at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433) at sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691) at sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91) at org.apache.nifi.processor.util.listen.dispatcher.DatagramChannelDispatcher.open(DatagramChannelDispatcher.java:99) at org.apache.nifi.processors.standard.ListenSyslog.onScheduled(ListenSyslog.java:322) ... 16 common frames omitted
... View more
Labels:
- Labels:
-
Apache NiFi
02-01-2018
10:11 PM
@Jay Kumar SenSharma I removed some proxy settings from /var/lib/ambari-server/ambari-env.sh and it worked 🙂 Thank you for your time and help!
... View more