Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi ConsumeKafka NullPointerException

Highlighted

NiFi ConsumeKafka NullPointerException

New Contributor

I want to read, with Consume_Kafka_2_0, all the headers that come with the message, but without knowing their names first.

 

In NiFi ConsumeKafka:

- Headers to Add as Attributes (Regex): knowheader1|knownHeader|knownHeaderN --> no problem

- Headers to Add as Attributes (Regex): .* --> at some point "Exception when processing data from kafka so will close the lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@xxxxxxx due to null: java.lang.NullPointerException

 

 

The above error seems due to the fact that some messages come with null headers, however I don't know how to save all the custom headers that come with the message and at the same time avoid the above error.

Do you know how to solve this problem?

2 REPLIES 2

Re: NiFi ConsumeKafka NullPointerException

Rising Star

Not sure if this fully answers your question, but I believe that in recent versions of Kafka (including 2.0) Null headers are not supported anymore. 

 

Also, you see a null pointer exception, but that is more a general indication that something went wrong, and does not directly indicate that the header is null. 

 

I would recommend you to find out what the header actually is (something null, or perhaps something complex) and search further from there. 

Re: NiFi ConsumeKafka NullPointerException

New Contributor

If you just want to write a message from Kafka to a file, is there a reason you can't just use ConsumeKafka -> PutFile?

The error is because you only put two Upsers lines inside of the if (flowFile != None): and if flow file is null you are still calling transfer on line 18 which is producing the error.

Don't have an account?
Coming from Hortonworks? Activate your account here