Support Questions

Find answers, ask questions, and share your expertise

Nifi processors freeze

avatar
New Contributor

Hi everybody,

 

Specifically, the nifi PublishMQTT and ListFile processors, sometimes are getting stuck, and they do not work again until they are stopped and runned again. The threads associated with the processors appear to be working but they don't do nothing. They are blocked. As a consequence, the data flow to the input of the MQTT processor is stopped.

 

Below, I copy some logs to help with the solution:

 

MQTT:

 

2021-01-29 05:47:58,317 ERROR [MQTT Snd: sender1] o.a.nifi.processors.mqtt.PublishMQTT PublishMQTT[id=9e5c9353-5eb3-3967-0856-e0a9f121257f] Connection to ssl://IP:port lost due to: Connection lost: Connection lost (32109) - java.net.SocketException: Connection reset by peer: socket write error
org.eclipse.paho.client.mqttv3.MqttException: Connection lost
at org.eclipse.paho.client.mqttv3.internal.CommsSender.handleRunException(CommsSender.java:194)
at org.eclipse.paho.client.mqttv3.internal.CommsSender.run(CommsSender.java:171)
at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at sun.security.ssl.SSLSocketOutputRecord.deliver(Unknown Source)
at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(Unknown Source)
at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
at java.io.BufferedOutputStream.write(Unknown Source)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttOutputStream.write(MqttOutputStream.java:85)
at org.eclipse.paho.client.mqttv3.internal.CommsSender.run(CommsSender.java:147)
... 1 common frames omitted

 


2021-01-29 05:47:58,317 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.mqtt.PublishMQTT PublishMQTT[id=9e5c9353-5eb3-3967-0856-e0a9f121257f] Failed to publish message.: Connection lost (32109) - java.net.SocketException: Connection reset by peer: socket write error
org.eclipse.paho.client.mqttv3.MqttException: Connection lost
at org.eclipse.paho.client.mqttv3.internal.CommsSender.handleRunException(CommsSender.java:194)
at org.eclipse.paho.client.mqttv3.internal.CommsSender.run(CommsSender.java:171)
at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at sun.security.ssl.SSLSocketOutputRecord.deliver(Unknown Source)
at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(Unknown Source)
at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
at java.io.BufferedOutputStream.write(Unknown Source)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttOutputStream.write(MqttOutputStream.java:85)
at org.eclipse.paho.client.mqttv3.internal.CommsSender.run(CommsSender.java:147)
... 1 common frames omitted

 

 

LISTFILE:

 

2021-01-29 03:30:09,054 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.ListFile ListFile[id=414e3fd5-dea7-356c-b41e-b94407465746] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: Failed to persist already-listed entities due to java.io.IOException: An established connection was aborted by the software in your host machine: org.apache.nifi.processor.exception.ProcessException: Failed to persist already-listed entities due to java.io.IOException: An established connection was aborted by the software in your host machine
org.apache.nifi.processor.exception.ProcessException: Failed to persist already-listed entities due to java.io.IOException: An established connection was aborted by the software in your host machine
at org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:342)
at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingEntities(AbstractListProcessor.java:818)
at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:432)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: An established connection was aborted by the software in your host machine
at sun.nio.ch.SocketDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
at sun.nio.ch.IOUtil.write(Unknown Source)
at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
at org.apache.nifi.remote.io.socket.SocketChannelOutputStream.write(SocketChannelOutputStream.java:94)
at java.io.BufferedOutputStream.write(Unknown Source)
at org.apache.nifi.remote.io.InterruptableOutputStream.write(InterruptableOutputStream.java:57)
at java.io.DataOutputStream.write(Unknown Source)
at java.io.ByteArrayOutputStream.writeTo(Unknown Source)
at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.serialize(DistributedMapCacheClientService.java:471)
at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.access$000(DistributedMapCacheClientService.java:55)
at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService$2.execute(DistributedMapCacheClientService.java:140)
at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.withCommsSession(DistributedMapCacheClientService.java:492)
at org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.put(DistributedMapCacheClientService.java:133)
at sun.reflect.GeneratedMethodAccessor79.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)
at com.sun.proxy.$Proxy101.put(Unknown Source)
at org.apache.nifi.processor.util.list.ListedEntityTracker.persistListedEntities(ListedEntityTracker.java:215)
at org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:340)
... 14 common frames omitted

 

 

Could someone guide me with the solution?

Thank you very much in advance,

Aitor

1 REPLY 1

avatar
Expert Contributor

Hi,

 

you can check your Nifi resources, specifically the Java Heap size found in "bootstrap.conf" file and increase that, please check this for Nifi performance best practices HDF/CFM NIFI Best practices for setting up a high ... - Cloudera Community . 

 

Best Regards,