Hi, I’m working with CDH 5.4.2 and we use flume for data aggregation.When we ran the flume agent, observed the following problem: 18:06:2015 11:26:17 ERROR [ChannelProcessor] Error while writing to required channel: org.apache.flume.channel.MemoryChannel{name: channel1}
java.lang.OutOfMemoryError: GC overhead limit exceeded Configuration File details: agent1.channels.channel1.type = memory
agent1.channels.channel1.capacity = 100000
agent1.channels.channel1.transactionCapactiy = 1000
agent1.sources.source1.channels = channel1
agent1.sinks = sink1
agent1.sinks.sink1.type = hdfs
agent1.sinks.sink1.channel = channel1
agent1.sinks.sink1.hdfs.useLocalTimeStamp = true
agent1.sinks.sink1.hdfs.path = hdfs://xxx/%Y%m%d%H
agent1.sinks.sink1.hdfs.filePrefix = xdr
agent1.sinks.sink1.hdfs.fileType = DataStream
agent1.sinks.sink1.hdfs.writeFormat = Text
agent1.sinks.sink1.hdfs.rollCount = 2000000
agent1.sinks.sink1.hdfs.rollSize = 0
agent1.sinks.sink1.hdfs.batchSize = 100 I really appreciate your help, on this issue. Regards, Osman Uygar
... View more
