Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

KafkaSource EXCEPTION, {} java.lang.NullPointerException

KafkaSource EXCEPTION, {} java.lang.NullPointerException

New Contributor

Hi all,

 

I developed a java program to send json files to a topic of kafka, the name of the topic is "portal". I create a consumer in kafka and a can see all  the files.

 

Now i want to ingest all de files in flume and send to a custom sink but i have the following errors.

 

2016-04-29 20:59:03,754 DEBUG org.apache.flume.source.kafka.KafkaSource: Waited: 1001
2016-04-29 20:59:03,754 DEBUG org.apache.flume.source.kafka.KafkaSource: Event #: 0
2016-04-29 20:59:03,754 ERROR org.apache.flume.source.kafka.KafkaSource: KafkaSource EXCEPTION, {}
java.lang.NullPointerException
        at org.apache.flume.instrumentation.MonitoredCounterGroup.increment(MonitoredCounterGroup.java:261)
        at org.apache.flume.instrumentation.kafka.KafkaSourceCounter.incrementKafkaEmptyCount(KafkaSourceCounter.java:49)
        at org.apache.flume.source.kafka.KafkaSource.process(KafkaSource.java:146)
        at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:139)
        at java.lang.Thread.run(Thread.java:745)
2016-04-29 20:59:05,164 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x5545d491fad0aa3 after 0ms
2016-04-29 20:59:07,167 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x5545d491fad0aa3 after 0ms
2016-04-29 20:59:08,765 DEBUG org.apache.flume.source.kafka.KafkaSource: Waited: 10
2016-04-29 20:59:08,765 DEBUG org.apache.flume.source.kafka.KafkaSource: Event #: 0
2016-04-29 20:59:08,775 DEBUG org.apache.flume.source.kafka.KafkaSource: Waited: 20
2016-04-29 20:59:08,775 DEBUG org.apache.flume.source.kafka.KafkaSource: Event #: 0

My flume configuration is

##Archivo de Configuracion Flume con Postgresql
# Naming the components on the current agent
Hive.sources = Twitter
Hive.channels = MemChannel5
Hive.sinks = ruckus

Hive.sources.Twitter.channels = MemChannel5

# Describing/Configuring the source
Hive.sources.Twitter.type = org.apache.flume.source.kafka.KafkaSource
Hive.sources.Twitter.zookeeperConnect = 192.168.1.153:2181
Hive.sources.Twitter.topic = portal
Hive.sources.Twitter.groupId = flume

Hive.sinks.ruckus.type = logger
Hive.sinks.ruckus.channel = MemChannel5
#Hive.sinks.ruckus.type = com.carlos.bigdata.sinks.JsonSink
#Hive.sinks.ruckus.hbaseZk = 192.168.1.150
#Hive.sinks.ruckus.hbaseZP = 2181
#Hive.sinks.ruckus.hbaseTabla = pb

# Describing/Configuring the channel
Hive.channels.MemChannel5.type = memory
Hive.channels.MemChannel5.capacity = 10000
Hive.channels.MemChannel5.transactionCapacity = 1000

can anybody help me?

 

Thank you!

 

1 REPLY 1

Re: KafkaSource EXCEPTION, {} java.lang.NullPointerException

Super Collaborator

What version of CDH are you using?  There is a known issue (https://issues.apache.org/jira/browse/FLUME-2578) in earlier versions of the flume kafka source that can cause the null pointer exception.

 

-pd