Member since
12-14-2015
21
Posts
5
Kudos Received
0
Solutions
06-22-2016
04:45 PM
Hi Mburgess, Thanks for your response.It will really helpful providing the information how to exclude the particular(xerces) jar at the time of NAR build. Thanks, Viswanatha Reddy
... View more
06-22-2016
01:19 PM
Please refer this screen shot for more information. capture.png
... View more
06-22-2016
01:12 PM
Hi Everyone, I'm facing the some weird issue. In my project we are using everything is custom code of NiFi, For xml parsing we are using the DOM parser, after deploying the NAR file into NiFi lib directory after restart i'm facing the below error. please anybody can give me the solution on this. Custom Processor: XmlValidation(Domparser) CustomPutHdfs(Hdfs) java.lang.ClassCastException: org.apache.xerces.parsers.XIncludeAwareParserConfiguration cannot be cast to org.apache.xerces.xni.parser.XMLParserConfiguration
at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2482) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2444) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2361) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.conf.Configuration.get(Configuration.java:968) ~[hadoop-common-2.6.2.jar:na]
at org.apache.nifi.hadoop.SecurityUtil.isSecurityEnabled(SecurityUtil.java:84) ~[nifi-hadoop-utils-0.6.0.jar:0.6.0]
at org.apache.nifi.hadoop.KerberosProperties.validatePrincipalAndKeytab(KerberosProperties.java:121) ~[nifi-hadoop-utils-0.6.0.jar:0.6.0]
at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.customValidate(AbstractHadoopProcessor.java:154) ~[nifi-hdfs-processors-0.6.0.jar:0.6.0]
at org.apache.nifi.components.AbstractConfigurableComponent.validate(AbstractConfigurableComponent.java:123) ~[nifi-api-0.6.0.jar:0.6.0]
at org.apache.nifi.controller.StandardProcessorNode.isValid(StandardProcessorNode.java:911) ~[nifi-framework-core-0.6.0.jar:0.6.0]
at org.apache.nifi.controller.FlowController.getProcessorStatus(FlowController.java:2529) [nifi-framework-core-0.6.0.jar:0.6.0]
at org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2146) [nifi-framework-core-0.6.0.jar:0.6.0]
at org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2162) [nifi-framework-core-0.6.0.jar:0.6.0]
at org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2162) [nifi-framework-core-0.6.0.jar:0.6.0]
at org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2162) [nifi-framework-core-0.6.0.jar:0.6.0]
at org.apache.nifi.controller.FlowController$HeartbeatMessageGeneratorTask.createHeartbeatMessage(FlowController.java:3819) [nifi-framework-core-0.6.0.jar:0.6.0]
at org.apache.nifi.controller.FlowController$HeartbeatMessageGeneratorTask.run(FlowController.java:3802) [nifi-framework-core-0.6.0.jar:0.6.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_71]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_71]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_71]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_71]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_71]
... View more
Labels:
- Labels:
-
Apache NiFi
05-31-2016
05:04 AM
Hi clark & Pierre, let me know the swap in swap out location of the queue. i mean where it will store the flowfiles at the time of queue have more than threshold value?
... View more
05-30-2016
02:04 PM
Thanks and sorry for asking these many questions. how do i setup the backpressure on source processor???
... View more
05-30-2016
01:42 PM
Hi @Pierre Villard, Thanks for inputs, for example i have a GetFile processor and PutFile processor, Lets consider GetFile having 2lakh files, i started both at one time,for some reason after processing 100 files PutFile gone to error state it is not taking any inputs. So GetFile writing the continuously write data into queue. here how long will keep hold the data in a queue. is there any loss of data when it cross the 20000 flow files, let's consider my PutFile error is resolved after one week. still data in queue???? Thanks for giving response.
... View more
05-30-2016
01:23 PM
1 Kudo
Labels:
- Labels:
-
Apache NiFi
01-29-2016
07:26 AM
1 Kudo
Hi Venkat, The problem is here you declared args parameter before the configuration you should declare the args after the configuration it won't give you an error. I tested it it's working fine.
<workflow-app xmlns="uri:oozie:workflow:0.2" name="pdr-distcp-wf"> <start to="distcp-node"/> <action name="distcp-node"> <distcp xmlns="uri:oozie:distcp-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode1}</name-node> <configuration> <property> <name>oozie.launcher.mapreduce.job.hdfs-servers</name> <value>${nameNode1},${nameNode2}</value> </property> </configuration> <arg>${SourceDir}</arg> <arg>${TargetDir}</arg> </distcp> <ok to="end"/> <error to="kill"/> </action> <kill name="kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>
... View more
12-14-2015
04:49 AM
Labels:
- Labels:
-
Apache Spark