07-08-2017 09:16 AM
how to deal with this error? Is the channel full?
clienta.properties
julyLog.sources= src1 julyLog.channels= chnl1 julyLog.sinks= snk1 #sources julyLog.sources.src1.type=spooldir julyLog.sources.src1.channels=chnl1 julyLog.sources.src1.spoolDir=/home/cloudera/sample-data/nasa_log/july julyLog.sources.src1.inputCharset=us-ascii #channels julyLog.channels.chnl1.type=memory julyLog.channels.chnl1.capacity=10000 julyLog.channels.chnl1.transactionCapacity=1000 #sink julyLog.sinks.snk1.type = avro julyLog.sinks.snk1.channel = chnl1 julyLog.sinks.snk1.hostname = localhost julyLog.sinks.snk1.port = 4545
below is the error
17/07/08 08:27:27 INFO sink.AvroSink: Attempting to create Avro Rpc client. 17/07/08 08:27:27 WARN api.NettyAvroRpcClient: Using default maxIOWorkers 17/07/08 08:27:28 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:27:32 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:27:35 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:27:39 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:27:42 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:27:46 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:27:47 ERROR flume.SinkRunner: Unable to deliver event. Exception follows. org.apache.flume.EventDeliveryException: Failed to send events at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Failed to send batch at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:315) at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376) ... 3 more Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Handshake timed out after 20000ms at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:359) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:303) ... 4 more Caused by: java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:201) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:357) ... 5 more 17/07/08 08:27:49 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:27:52 INFO sink.AbstractRpcSink: Rpc sink snk1: Building RpcClient with hostname: localhost, port: 4545 17/07/08 08:27:52 INFO sink.AvroSink: Attempting to create Avro Rpc client. 17/07/08 08:27:52 WARN api.NettyAvroRpcClient: Using default maxIOWorkers 17/07/08 08:27:53 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:27:56 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:00 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:03 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:07 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:10 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:12 ERROR flume.SinkRunner: Unable to deliver event. Exception follows. org.apache.flume.EventDeliveryException: Failed to send events at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Failed to send batch at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:315) at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376) ... 3 more Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Handshake timed out after 20000ms at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:359) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:303) ... 4 more Caused by: java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:201) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:357) ... 5 more 17/07/08 08:28:14 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:17 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:17 INFO sink.AbstractRpcSink: Rpc sink snk1: Building RpcClient with hostname: localhost, port: 4545 17/07/08 08:28:17 INFO sink.AvroSink: Attempting to create Avro Rpc client. 17/07/08 08:28:17 WARN api.NettyAvroRpcClient: Using default maxIOWorkers 17/07/08 08:28:21 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:24 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:28 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:31 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:35 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:37 ERROR flume.SinkRunner: Unable to deliver event. Exception follows. org.apache.flume.EventDeliveryException: Failed to send events at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Failed to send batch at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:315) at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376) ... 3 more Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Handshake timed out after 20000ms at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:359) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:303) ... 4 more Caused by: java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:201) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:357) ... 5 more 17/07/08 08:28:38 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:42 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:42 INFO sink.AbstractRpcSink: Rpc sink snk1: Building RpcClient with hostname: localhost, port: 4545 17/07/08 08:28:42 INFO sink.AvroSink: Attempting to create Avro Rpc client. 17/07/08 08:28:42 WARN api.NettyAvroRpcClient: Using default maxIOWorkers 17/07/08 08:28:45 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:49 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:52 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:28:56 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:28:59 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:29:02 ERROR flume.SinkRunner: Unable to deliver event. Exception follows. org.apache.flume.EventDeliveryException: Failed to send events at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Failed to send batch at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:315) at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376) ... 3 more Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Handshake timed out after 20000ms at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:359) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:303) ... 4 more Caused by: java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:201) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:357) ... 5 more 17/07/08 08:29:03 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:29:06 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:29:07 INFO sink.AbstractRpcSink: Rpc sink snk1: Building RpcClient with hostname: localhost, port: 4545 17/07/08 08:29:07 INFO sink.AvroSink: Attempting to create Avro Rpc client. 17/07/08 08:29:07 WARN api.NettyAvroRpcClient: Using default maxIOWorkers 17/07/08 08:29:10 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:29:13 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:29:17 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:29:20 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:29:24 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:29:27 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:29:27 ERROR flume.SinkRunner: Unable to deliver event. Exception follows. org.apache.flume.EventDeliveryException: Failed to send events at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Failed to send batch at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:315) at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376) ... 3 more Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Handshake timed out after 20000ms at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:359) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:303) ... 4 more Caused by: java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:201) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:357) ... 5 more 17/07/08 08:29:31 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:29:32 INFO sink.AbstractRpcSink: Rpc sink snk1: Building RpcClient with hostname: localhost, port: 4545 17/07/08 08:29:32 INFO sink.AvroSink: Attempting to create Avro Rpc client. 17/07/08 08:29:32 WARN api.NettyAvroRpcClient: Using default maxIOWorkers 17/07/08 08:29:34 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:29:38 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:29:41 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:29:45 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:29:48 WARN source.SpoolDirectorySource: The channel is full, and cannot write data now. The source will try again after 4000 milliseconds 17/07/08 08:29:52 INFO avro.ReliableSpoolingFileEventReader: Last read was never committed - resetting mark position. 17/07/08 08:29:52 ERROR flume.SinkRunner: Unable to deliver event. Exception follows. org.apache.flume.EventDeliveryException: Failed to send events at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 4545 }: Failed to send batch at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:315) at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376)