Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi ConsumeKafka NullPointerException

NiFi ConsumeKafka NullPointerException

New Contributor

I want to read, with Consume_Kafka_2_0, all the headers that come with the message, but without knowing their names first.

 

In NiFi ConsumeKafka:

- Headers to Add as Attributes (Regex): knowheader1|knownHeader|knownHeaderN --> no problem

- Headers to Add as Attributes (Regex): .* --> at some point "Exception when processing data from kafka so will close the lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@xxxxxxx due to null: java.lang.NullPointerException

 

 

The above error seems due to the fact that some messages come with null headers, however I don't know how to save all the custom headers that come with the message and at the same time avoid the above error.

Do you know how to solve this problem?

2 REPLIES 2
Highlighted

Re: NiFi ConsumeKafka NullPointerException

Expert Contributor

Not sure if this fully answers your question, but I believe that in recent versions of Kafka (including 2.0) Null headers are not supported anymore. 

 

Also, you see a null pointer exception, but that is more a general indication that something went wrong, and does not directly indicate that the header is null. 

 

I would recommend you to find out what the header actually is (something null, or perhaps something complex) and search further from there. 


If this answer helped, please mark it as 'solved' and/or if it is valuable for future readers please apply 'kudos'.
Highlighted

Re: NiFi ConsumeKafka NullPointerException

New Contributor

If you just want to write a message from Kafka to a file, is there a reason you can't just use ConsumeKafka -> PutFile?

The error is because you only put two Upsers lines inside of the if (flowFile != None): and if flow file is null you are still calling transfer on line 18 which is producing the error.

Don't have an account?
Coming from Hortonworks? Activate your account here