question send kafka topic as header when publishing data to another topic in NIFI in Support Questions

send kafka topic as header when publishing data to another topic in NIFI

Rohit1997jio — Fri, 24 Nov 2023 09:04:10 GMT

i want to send value of kafka.topic as header i.e (the topic from where i consumed message) to publish topic (another topic) by using publish processor.

But iam getting below error in publish processor-

Re: send kafka topic as header when publishing data to another topic in NIFI

joseomjr — Fri, 24 Nov 2023 17:53:57 GMT

The error tells you it needs to be a regular expression matching the attribute you wish to send so if it's kafka.topic it could be that literally or more accurately would be kafka\.topic

Re: send kafka topic as header when publishing data to another topic in NIFI

Rohit1997jio — Sun, 26 Nov 2023 16:39:32 GMT

kafka.topic is a key , it will have value of a kafka topic name e.g abc_xyz .

this value will keep on depending from where iam consuming the meassge , from which topic .

my requirement is to send the topic from where i consumed as header to the topic where iam publishing it .

how can i do that ??

Re: send kafka topic as header when publishing data to another topic in NIFI

joseomjr — Mon, 27 Nov 2023 01:54:24 GMT

Yes, the regular expression it's asking for is that of the attribute containing the value you want... It's not asking for the value. Since you want the value of the attribute kafka.topic (name/key) then this is what the regular expression you need to match. Have you tried just putting what I previously recommended "kafka\.topic"?

Re: send kafka topic as header when publishing data to another topic in NIFI

Rohit1997jio — Tue, 28 Nov 2023 13:45:48 GMT

kafka\.topic , i tried adding this , but it did not work . I checked kafka headers too there also nothing being printed in haeder.

Re: send kafka topic as header when publishing data to another topic in NIFI

steven-matison — Tue, 28 Nov 2023 13:47:01 GMT

@joseomjr Is on to the right solution here. Your regex statement should match "kafka.topic" not "${kafka.topic}". A quick test in regex101.com confirms "kafka\.topic" should match.