Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Second tier Flume agent not binding to first tier

Second tier Flume agent not binding to first tier

Explorer

Hi Everyone,

 

I am trying to setup a simple two tier flume agents using the Cloudera VM. 

The code is as follow:

# First-tier agent

agent1.sources = source1 
agent1.sinks = sink1
agent1.channels = channel1

agent1.sources.source1.channels = channel1 
agent1.sinks.sink1.channel = channel1 

agent1.sources.source1.type = spooldir 
agent1.sources.source1.spoolDir = /tmp/spooldir 

agent1.sinks.sink1.type = avro 
agent1.sinks.sink1.hostname = localhost 
agent1.sinks.sink1.port = 10000

agent1.channels.channel1.type = file
agent1.channels.channel1.checkpointDir = /tmp/flume/channel1/check
agent1.channels.channel1.dataDirs = /tmp/flume/channel1/data

# Second-tier agent

agent2.sources = source2 
agent2.sinks = sink2
agent2.channels = channel2

agent2.sources.source2.channels = channel2
agent2.sinks.sink2.channel = channel2

agent2.sources.source2.type = avro 
agent2.sources.source2.bind = localhost 
agent2.sources.source2.port = 10000

agent2.sinks.sink2.type = hdfs 
agent2.sinks.sink2.hdfs.path = /tmp/flume
agent2.sinks.sink2.hdfs.filePrefix = events
agent2.sinks.sink2.hdfs.fileSuffix = .log
agent2.sinks.sink2.hdfs.fileType = DataStream

agent2.channels.channel2.type = file
agent2.channels.channel2.checkpointDir = /tmp/flume/channel2/check
agent2.channels.channel2.dataDirs = /tmp/flume/channel2/data

The first agent looks OK but as soon as I run the second agent, its source cannot connect to the first agent's sink.

It gives the following error: 

17/10/06 14:35:33 INFO node.Application: Starting Sink sink2
17/10/06 14:35:33 INFO node.Application: Starting Source source2
17/10/06 14:35:33 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: SINK, name: sink2: Successfully registered new MBean.
17/10/06 14:35:33 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: sink2 started
17/10/06 14:35:33 INFO source.AvroSource: Starting Avro source source2: { bindAddress: localhost, port: 10000 }...
17/10/06 14:35:35 ERROR lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner: { source:Avro source source2: { bindAddress: localhost, port: 10000 } } - Exception follows.
org.jboss.netty.channel.ChannelException: Failed to bind to: localhost/127.0.0.1:10000
:10000
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
	at org.apache.flume.source.AvroSource.start(AvroSource.java:242)
	at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
	at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:444)
	at sun.nio.ch.Net.bind(Net.java:436)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:372)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:296)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)

Does anybbody have any idea why the second flume agent fails to bind? 

 

Thanks.

 

Don't have an account?
Coming from Hortonworks? Activate your account here