Member since
03-06-2017
45
Posts
4
Kudos Received
0
Solutions
05-02-2022
08:41 AM
Could you please share the steps to configure Nifi 0.x baseline
... View more
05-11-2021
05:09 AM
1 Kudo
Thanks @VidyaSargur - I just started a new thread, per your suggestion.
... View more
05-21-2020
04:55 AM
@myoung ...in nifi i have used invoked http processor that get data from web app.. i want to apply the condition on json data file ...i means when the resource="abc" the alarm get filter and goes to one processor and other alarm will go to another processor.....so my question is what are the processor is required..as of now i have used one http invoked that will provide all the alarm form my web app ..two putfile processor for storing alarm one for resource="abc" and one for other resource. and i have also used one Routonatrribute processor for condition..in RoutOnAttribute config i have decalred on dynamic property whic is ....the name of property is alarms.resource and the value is ${alarms.resource:equals('abc')} the below is respon of Invoked http processor...want the alarm that contain resource='abc' that will go to one put file processor { "alamrs": [ { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }, { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }, { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , } @
... View more
05-09-2018
07:28 PM
@Prakhar Agrawal Then your issue is going to be the backend validation that is going on. How much validation that is occurring depends on what you are doing. Shu's recommendation for disabling processors that are not running should help a great deal. - There is no such thing as "Max Cron Driven Thread Count". I am going to assume you meant "Max Event Driven Thread count." There is nothing you will add to your canvas that will use the "Event Driven" scheduling strategy. You physically need to change the configuration on a component to use "Event Driven". Event Driven is consider experimental and deprecated in favor of advance made with the Timer driven Scheduling strategy. By setting "Max Event Driven Thread count" to 50 you are creating a pool of 50 reserved threads. I recommend not using "Event Driven" Scheduling strategy anywhere and reducing the max Event Driven thread count setting to "1". Reducing the "Max Event driven thread count" setting will require a NiFi restart.
... View more
05-04-2018
03:57 PM
@Prakhar Agrawal @Felix Albani is correct. There is no way to automatically have a node delete his flow.xml.gz in favor of the clusters flow. If we allowed that it could lead to unexpected data loss. Lets assume a node was taken out of the cluster do perform some side work and the user tries to rejoin it to cluster, if it just took the clusters flow, any data queued in a connection that doe snot exist in clusters flow would be lost. It would be impossible for Nifi to know if the joining of this node to this cluster was a mistake or intended, so NiFi simply informs you there is a mismatch and expects you to resolve the issue. - Also noticed you mentioned "NCM" (NiFi Cluster Manager). NIFi moved away from having a NCM staring with Apache NIFi 1.x version. Newer version have a zero master cluster where any connected node can be elected as the cluster's coordinator. - Thanks, Matt
... View more
04-24-2018
01:52 PM
@Prakhar Agrawal Connect with "Group1_Port1" input port that are also placed inside my "Group1" process group? It's not possible because Group_port1 is an input port inside processor group but Root_port1 is an remote processor group because it is on root canvas level. Root_port1,Root_port2 is used to distribute the data across NiFi instance that's the reason why you are able to see all the root canvas input ports(Root_port1,Root_port2) in the drop down list. Group_port1 is an input port which is used to transfer the data into process group. Please refer to this link to understand more about process groups and remote process groups.
... View more
07-11-2017
08:01 AM
Sorry....I attached the wrong code...please find it here nodereadfrommongo.txt
... View more
06-14-2017
11:22 AM
@Prakhar Agrawal Take a look at the DistributedMapCache. It allows you to save flowfile data for consumption by processors across nodes and processors. There are two controller components, the DistributesMapCacheServer (runs on one node) and the DistricutedMapCacheClient (runs on all nodes if you've got a cluster). There are also the processors that interact with the DistributedMapCache, namely FetchDistributedMapCache and PutDistributedMapCache Take a look here for some elaboration/documentation: https://nifi.apache.org/docs.html https://community.hortonworks.com/questions/35223/distributedmapcacheclientservice-nifi-wecrawlerxml.html Here for examples using DistributedMapCache: https://github.com/hortonworks-gallery/nifi-templates/tree/master/templates (webcrawler.xml template) http://funnifi.blogspot.ca/2016/04/
... View more
06-09-2017
02:31 AM
@Prakhar Agrawal One of the reason could be the customprocessor is not stopped; your custom code is blocking and does not react to interrupts and hence you are not able to restart. One of the solution could be restart of Apache Nifi
... View more
05-29-2017
03:15 PM
Thanks Timothy I have find out the way to read template's xml file in onTrigger method of my custom processor, but I am getting some issues. I am using getTemplates() method of StandardNiFiServiceFacade class that implements NiFiServiceFacade interface, and I make object of that interface like this -- private NiFiServiceFacade serviceFacade = new StandardNiFiServiceFacade(); in my custom processor's code. But when I am going to up my nifi server, I got error on this line. Below are the error I am getting: java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.hadoop.SparkConnector could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[na:1.8.0_111]
at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[na:1.8.0_111]
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[na:1.8.0_111]
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[na:1.8.0_111]
at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[na:1.8.0_111]
at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:116) ~[nifi-nar-utils-1.1.2.jar:1.1.2]
at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:97) ~[nifi-nar-utils-1.1.2.jar:1.1.2]
at org.apache.nifi.NiFi.<init>(NiFi.java:139) ~[nifi-runtime-1.1.2.jar:1.1.2]
at org.apache.nifi.NiFi.main(NiFi.java:262) ~[nifi-runtime-1.1.2.jar:1.1.2] Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/web/revision/RevisionClaim
at org.apache.nifi.processors.hadoop.SparkConnector.<init>(SparkConnector.java:53) ~[nifi-hdfs-processors-1.1.2.jar:1.1.2]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_111]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_111]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_111]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_111]
at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_111]
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[na:1.8.0_111]
... 6 common frames omitted Caused by: java.lang.ClassNotFoundException: org.apache.nifi.web.revision.RevisionClaim
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_111]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_111]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_111]
... 13 common frames omitted
I wont able to understand what exactly is the problem, as I have already added all required jar files in nar. Still I am getting this error. Please suggest me what I am doing wrong here or tell me some alternate way if you know that how I get all template object in TemplateEntity class in my onTrigger method of customProcessor. Thanks in advance
... View more