question Re: NiFi 1.3 - PublishKafka_0_10 - A message in the stream exceeds the maximum allowed message size of 1048576 bytes in Support Questions https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/205740#M167706 <P>Thanks this solution worked. </P> Wed, 22 Nov 2017 02:24:33 GMT mayank_rathi 2017-11-22T02:24:33Z NiFi 1.3 - PublishKafka_0_10 - A message in the stream exceeds the maximum allowed message size of 1048576 bytes https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/205738#M167704 <DIV> <DIV> <DIV>Hello All,I am getting this error in PublishKafka_0_10 processor for a message of size 2.08 MB. I have updated Max Request Size to 10 MB in processor properties and max.request.size to 10 MB in Kafka's server.properties. After reboot Kafka Broker I can see that max.request.size = 10 MB in Kafka logs but I am still getting below error. What am I missing here?<BR /><BR />2017-11-17 11:07:47,966 ERROR [Timer-Driven Process Thread-4] o.a.n.p.kafka.pubsub.PublishKafka_0_10 PublishKafka_0_10[id=e6d932d9-97ae-1647-aa8f-86d07791ce25] Failed to send all message for StandardFlowFileRecord[uuid=fa2399e5-bea5-4113-b58b-6cdef228733c,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1510934860019-132, container=default, section=132], offset=0, length=2160613],offset=0,name=12337127439954063,size=2160613] to Kafka; routing to failure due to <A href="http://org.apache.nifi.stream.io">org.apache.nifi.stream.io</A>.exception.TokenTooLargeException: A message in the stream exceeds the maximum allowed message size of 1048576 bytes.: {}<BR /><A href="http://org.apache.nifi.stream.io">org.apache.nifi.stream.io</A>.exception.TokenTooLargeException: A message in the stream exceeds the maximum allowed message size of 1048576 bytes.<BR /> at <A href="http://org.apache.nifi.stream.io">org.apache.nifi.stream.io</A>.util.AbstractDemarcator.extractDataToken(AbstractDemarcator.java:157)<BR /> at <A href="http://org.apache.nifi.stream.io">org.apache.nifi.stream.io</A>.util.StreamDemarcator.nextToken(StreamDemarcator.java:129)<BR /> at org.apache.nifi.processors.kafka.pubsub.PublisherLease.publish(PublisherLease.java:78)<BR /> at org.apache.nifi.processors.kafka.pubsub.PublishKafka_0_10$1.process(PublishKafka_0_10.java:334)<BR /> at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2136)<BR /> at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2106)<BR /> at org.apache.nifi.processors.kafka.pubsub.PublishKafka_0_10.onTrigger(PublishKafka_0_10.java:330)<BR /> at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)<BR /> at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)<BR /> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)<BR /> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)<BR /> at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)<BR /> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)<BR /> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)<BR /> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)<BR /> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)<BR /> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)<BR /> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)<BR /> at java.lang.Thread.run(Thread.java:748)Thanks and Regards</DIV></DIV></DIV> Sat, 18 Nov 2017 01:01:08 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/205738#M167704 mayank_rathi 2017-11-18T01:01:08Z Re: NiFi 1.3 - PublishKafka_0_10 - A message in the stream exceeds the maximum allowed message size of 1048576 bytes https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/205739#M167705 <P>Verify whether message.max.bytes (or max.message.bytes for the topic) is set to an appropriate value to support the large messages. And to enable the consumers to read from this topic, set fetch.message.max.bytes as well.</P> Tue, 21 Nov 2017 12:48:43 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/205739#M167705 rmaruthiyodan 2017-11-21T12:48:43Z Re: NiFi 1.3 - PublishKafka_0_10 - A message in the stream exceeds the maximum allowed message size of 1048576 bytes https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/205740#M167706 <P>Thanks this solution worked. </P> Wed, 22 Nov 2017 02:24:33 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/205740#M167706 mayank_rathi 2017-11-22T02:24:33Z Re: NiFi 1.3 - PublishKafka_0_10 - A message in the stream exceeds the maximum allowed message size of 1048576 bytes https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/295271#M217653 <P>Hi , where did you found the property he talked about to modify it ? because I'm facing the same issue ? thank in advance</P> Fri, 01 May 2020 13:29:03 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-1-3-PublishKafka-0-10-A-message-in-the-stream-exceeds/m-p/295271#M217653 AndyAjram 2020-05-01T13:29:03Z