Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

KafkaSource EXCEPTION, {} java.lang.NullPointerException

KafkaSource EXCEPTION, {} java.lang.NullPointerException

New Contributor

Hi all,


I developed a java program to send json files to a topic of kafka, the name of the topic is "portal". I create a consumer in kafka and a can see all  the files.


Now i want to ingest all de files in flume and send to a custom sink but i have the following errors.


2016-04-29 20:59:03,754 DEBUG org.apache.flume.source.kafka.KafkaSource: Waited: 1001
2016-04-29 20:59:03,754 DEBUG org.apache.flume.source.kafka.KafkaSource: Event #: 0
2016-04-29 20:59:03,754 ERROR org.apache.flume.source.kafka.KafkaSource: KafkaSource EXCEPTION, {}
        at org.apache.flume.instrumentation.MonitoredCounterGroup.increment(
        at org.apache.flume.instrumentation.kafka.KafkaSourceCounter.incrementKafkaEmptyCount(
        at org.apache.flume.source.kafka.KafkaSource.process(
        at org.apache.flume.source.PollableSourceRunner$
2016-04-29 20:59:05,164 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x5545d491fad0aa3 after 0ms
2016-04-29 20:59:07,167 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x5545d491fad0aa3 after 0ms
2016-04-29 20:59:08,765 DEBUG org.apache.flume.source.kafka.KafkaSource: Waited: 10
2016-04-29 20:59:08,765 DEBUG org.apache.flume.source.kafka.KafkaSource: Event #: 0
2016-04-29 20:59:08,775 DEBUG org.apache.flume.source.kafka.KafkaSource: Waited: 20
2016-04-29 20:59:08,775 DEBUG org.apache.flume.source.kafka.KafkaSource: Event #: 0

My flume configuration is

##Archivo de Configuracion Flume con Postgresql
# Naming the components on the current agent
Hive.sources = Twitter
Hive.channels = MemChannel5
Hive.sinks = ruckus

Hive.sources.Twitter.channels = MemChannel5

# Describing/Configuring the source
Hive.sources.Twitter.type = org.apache.flume.source.kafka.KafkaSource
Hive.sources.Twitter.zookeeperConnect =
Hive.sources.Twitter.topic = portal
Hive.sources.Twitter.groupId = flume

Hive.sinks.ruckus.type = logger = MemChannel5
#Hive.sinks.ruckus.type = com.carlos.bigdata.sinks.JsonSink
#Hive.sinks.ruckus.hbaseZk =
#Hive.sinks.ruckus.hbaseZP = 2181
#Hive.sinks.ruckus.hbaseTabla = pb

# Describing/Configuring the channel
Hive.channels.MemChannel5.type = memory
Hive.channels.MemChannel5.capacity = 10000
Hive.channels.MemChannel5.transactionCapacity = 1000

can anybody help me?


Thank you!



Re: KafkaSource EXCEPTION, {} java.lang.NullPointerException

Super Collaborator

What version of CDH are you using?  There is a known issue ( in earlier versions of the flume kafka source that can cause the null pointer exception.