Member since
07-03-2017
48
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
929 | 07-17-2017 11:34 PM |
11-21-2017
06:24 PM
Thanks this solution worked.
... View more
11-17-2017
05:01 PM
Hello All,I am getting this error in
PublishKafka_0_10 processor for a message of size 2.08 MB. I have
updated Max Request Size to 10 MB in processor properties and
max.request.size to 10 MB in Kafka's server.properties. After reboot
Kafka Broker I can see that max.request.size = 10 MB in Kafka logs but I
am still getting below error. What am I missing here? 2017-11-17 11:07:47,966 ERROR [Timer-Driven Process Thread-4] o.a.n.p.kafka.pubsub.PublishKafka_0_10 PublishKafka_0_10[id=e6d932d9-97ae-1647-aa8f-86d07791ce25] Failed to send all message for StandardFlowFileRecord[uuid=fa2399e5-bea5-4113-b58b-6cdef228733c,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1510934860019-132, container=default, section=132], offset=0, length=2160613],offset=0,name=12337127439954063,size=2160613] to Kafka; routing to failure due to org.apache.nifi.stream.io.exception.TokenTooLargeException: A message in the stream exceeds the maximum allowed message size of 1048576 bytes.: {} org.apache.nifi.stream.io.exception.TokenTooLargeException: A message in the stream exceeds the maximum allowed message size of 1048576 bytes. at org.apache.nifi.stream.io.util.AbstractDemarcator.extractDataToken(AbstractDemarcator.java:157) at org.apache.nifi.stream.io.util.StreamDemarcator.nextToken(StreamDemarcator.java:129) at org.apache.nifi.processors.kafka.pubsub.PublisherLease.publish(PublisherLease.java:78) at org.apache.nifi.processors.kafka.pubsub.PublishKafka_0_10$1.process(PublishKafka_0_10.java:334) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2136) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2106) at org.apache.nifi.processors.kafka.pubsub.PublishKafka_0_10.onTrigger(PublishKafka_0_10.java:330) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)Thanks and Regards
... View more
Labels:
- Labels:
-
Apache NiFi
10-18-2017
04:22 PM
Thanks Matt.
... View more
10-12-2017
06:20 PM
Hello All,Can someone please help in understanding what I am doing wrong here?I am passing following values to PutSQL processor sql.args.1.type 2005 sql.args.1.value V99|1.0|
E99|BILLING|1234567|11049532|350|201706|201709|
R03|350|11049532|4| |KK|174|A|07/27/1948| |322| |0|F| | |201712|201709|0|168900.56|0|0|0|0|40134|201712|0|0|316| |200402|0| | | | | | | | | |POPR_CUPDT| | | | | | | | | | | |0|0|2|200402|0| | | |
P03|0|1|0|0|199404|317|201711|08/17/2017| |0|201712|08/22/2017|322|0|
K06|201706|111.53|111.53|14.8|96.73|111.53|15||0|0|
K06|201707|117.48|117.48|15.85|101.63|117.48|15||0|0|
K06|201708|227.98|227.98|44.34|183.64|227.98|30||0|0|
SQL query is: select obds_loader.getmain(?) from dual PutSQL processor is throwing below error. 2017-10-12 13:56:39,129 ERROR [Timer-Driven Process Thread-1] o.apache.nifi.processors.standard.PutSQL PutSQL[id=0beb1a31-980d-38f9-fc0e-6c0ac7a68984]
Failed to update database due to a failed batch update,
java.sql.BatchUpdateException: invalid batch command. There were a total
of 1 FlowFiles that failed, 0 that succeeded, and 0 that were not
execute and will be routed to retry; : java.sql.BatchUpdateException:
invalid batch command java.sql.BatchUpdateException: invalid batch command at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:10358) at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:230) at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297) at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297) at org.apache.nifi.processors.standard.PutSQL.lambda$null$10(PutSQL.java:348) at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127) at org.apache.nifi.processors.standard.PutSQL.lambda$new$12(PutSQL.java:346) at org.apache.nifi.processor.util.pattern.PutGroup.putFlowFiles(PutGroup.java:94) at org.apache.nifi.processor.util.pattern.Put.onTrigger(Put.java:101) at org.apache.nifi.processors.standard.PutSQL.lambda$onTrigger$20(PutSQL.java:554) at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114) at org.apache.nifi.processor.util.pattern.RollbackOnFailure.onTrigger(RollbackOnFailure.java:184) at org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:554) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Thanks and Regards
... View more
Labels:
- Labels:
-
Apache NiFi
09-15-2017
01:33 AM
Thank you all. All 3 answers were correct and pointed me in right direction.
... View more
09-14-2017
06:35 PM
Hello All, I am getting below error while installing Ambari. How do I resolve this? Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? n Configuring database... Default properties detected. Using built-in database. Configuring ambari database... Checking PostgreSQL... Running initdb: This may take up to a minute. About to start PostgreSQL [sudo] password for kfkadm: ERROR: Exiting with exit code 3. REASON: Unable to start PostgreSQL server. Exiting I had installed PostgreSQL by running below command: sudo
yum install https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rpm and then ran following command to Setup Ambari: sudo
yum install ambari-server Thanks!!
... View more
Labels:
- Labels:
-
Apache Ambari
09-05-2017
11:06 PM
Hello All, Logged https://issues.apache.org/jira/browse/NIFI-4352 for this issue. Thanks!!
... View more
09-05-2017
08:33 PM
Thanks. My issue is resolved now.
... View more
09-05-2017
04:14 PM
Even I am exploring ways to deliver data in order via NIFi. Please note that even on single node your data can get out of order if NiFi penalizes a fiow file or if you try to implement some custom "retry" logic.
... View more
09-04-2017
07:31 PM
Hello All, I dragged an Output Port in canvas inside a Process Group and I am getting error that - Port X is invalid because Output Connection for port X is not defined. What configuration am I missing? Thanks!!
... View more
Labels:
- Labels:
-
Apache NiFi