Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi ExecuteFlumeSink with ClosedChannelException

NiFi ExecuteFlumeSink with ClosedChannelException

New Contributor

Hello,
I'm new to NiFi(HDF V3.0.0.0, NiFi Version 1.2.0.3.0.0.0-453). I use ExecuteFlumeSink to put some messages to HDFS. Some of the ExecuteFlumeSink processors always throw the following Exception after they are started for a few hours and the Exception keeps be throwing until I restart the processors.

2017-11-14 12:48:38,525 ERROR [Timer-Driven Process Thread-12] o.a.n.processors.flume.ExecuteFlumeSink ExecuteFlumeSink[id=c4453fbe-0afe-1db0-b3d7- 1504c352c0bc] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: Flume event delivery failed: {} org.apache.nifi.processor.exception.ProcessException: Flume event delivery failed at org.apache.nifi.processors.flume.ExecuteFlumeSink.onTrigger(ExecuteFlumeSink.java:145) at org.apache.nifi.processors.flume.AbstractFlumeProcessor.onTrigger(AbstractFlumeProcessor.java:148) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.flume.EventDeliveryException: org.apache.flume.auth.SecurityException: Privileged action failed at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:463) at org.apache.nifi.processors.flume.ExecuteFlumeSink.onTrigger(ExecuteFlumeSink.java:143) ... 12 common frames omitted Caused by: org.apache.flume.auth.SecurityException: Privileged action failed at org.apache.flume.auth.UGIExecutor.execute(UGIExecutor.java:49) at org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:676) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: java.nio.channels.ClosedChannelException: null at org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1546) at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:104) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58) at java.io.DataOutputStream.write(DataOutputStream.java:107) at java.io.FilterOutputStream.write(FilterOutputStream.java:97) at org.apache.flume.serialization.BodyTextEventSerializer.write(BodyTextEventSerializer.java:71) at org.apache.flume.sink.hdfs.HDFSDataStream.append(HDFSDataStream.java:124) at org.apache.flume.sink.hdfs.BucketWriter$7.call(BucketWriter.java:550) at org.apache.flume.sink.hdfs.BucketWriter$7.call(BucketWriter.java:547) at org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:679) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.flume.auth.UGIExecutor.execute(UGIExecutor.java:47) ... 5 common frames omitted 2017-11-14 12:48:38,525 ERROR [Timer-Driven Process Thread-12] o.a.flume.sink.hdfs.AbstractHDFSWriter Unexpected error while checking replication f actor java.lang.reflect.InvocationTargetException: null at sun.reflect.GeneratedMethodAccessor436.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.flume.sink.hdfs.AbstractHDFSWriter.getNumCurrentReplicas(AbstractHDFSWriter.java:165) at org.apache.flume.sink.hdfs.AbstractHDFSWriter.isUnderReplicated(AbstractHDFSWriter.java:84) at org.apache.flume.sink.hdfs.BucketWriter.shouldRotate(BucketWriter.java:583) at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:518) at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:418) at org.apache.nifi.processors.flume.ExecuteFlumeSink.onTrigger(ExecuteFlumeSink.java:143) at org.apache.nifi.processors.flume.AbstractFlumeProcessor.onTrigger(AbstractFlumeProcessor.java:148) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.nio.channels.ClosedChannelException: null at org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1546) at org.apache.hadoop.hdfs.DFSOutputStream.getCurrentBlockReplication(DFSOutputStream.java:2099) at org.apache.hadoop.hdfs.DFSOutputStream.getNumCurrentReplicas(DFSOutputStream.java:2088) ... 21 common frames omitted 2017-11-14 12:48:38,526 ERROR [Timer-Driven Process Thread-12] org.apache.flume.sink.hdfs.HDFSEventSink process failed org.apache.flume.auth.SecurityException: Privileged action failed at org.apache.flume.auth.UGIExecutor.execute(UGIExecutor.java:49) at org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:676) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.nio.channels.ClosedChannelException: null at org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1546) at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:104) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58) at java.io.DataOutputStream.write(DataOutputStream.java:107) at java.io.FilterOutputStream.write(FilterOutputStream.java:97) at org.apache.flume.serialization.BodyTextEventSerializer.write(BodyTextEventSerializer.java:71) at org.apache.flume.sink.hdfs.HDFSDataStream.append(HDFSDataStream.java:124) at org.apache.flume.sink.hdfs.BucketWriter$7.call(BucketWriter.java:550) at org.apache.flume.sink.hdfs.BucketWriter$7.call(BucketWriter.java:547) at org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:679) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.flume.auth.UGIExecutor.execute(UGIExecutor.java:47) ... 5 common frames omitted

And this is the ExecuteFlumeSink settings for one of these processors.

42577-rbbmq1okfjmabm4vaagpkuoaele836.png

Is there any way to fix this?
Thanks!

Don't have an account?
Coming from Hortonworks? Activate your account here