question Re: PublishKafkaProcessor , request is not going to failure relation when kafka broker value is wrong in Support Questions

PublishKafkaProcessor , request is not going to failure relation when kafka broker value is wrong

Rohit1997jio — Tue, 26 Dec 2023 17:05:56 GMT

In my flow iam publishing data to kafka topic , if there is failure , that request should go to failure relation where iam publishing that data to error topic , but in my case when there is not connectivity with kafka or kafka broker value is not correct request is getting queue in PublishKafka processor not moving to failure relation .

This queue will effect my other requests any even my error handling logic .Please suggest some solution

Attaching the image of error and my nifi flow-

Re: PublishKafkaProcessor , request is not going to failure relation when kafka broker value is wrong

joseomjr — Wed, 27 Dec 2023 22:01:39 GMT

PublishKafka must have an active connection to Kafka before it even attempts to send a FlowFile which mean it never even gets into the block of code that sends it and routes it to "success" or "failure" accordingly. Making sure your Kafka cluster is up and running should be the focus if this is what you're experiencing. My guess if this "error" topic you have is on the same Kafka cluster then, even if PublishKafka was able to route a FlowFile to "failure" when it's unable to connect to Kafka, it wouldn't work anyways.

Re: PublishKafkaProcessor , request is not going to failure relation when kafka broker value is wrong

Rohit1997jio — Thu, 28 Dec 2023 06:46:21 GMT

My "error" topic is on different Kafka cluster . So i wanted my data which is failing due to connection error should move to failure and then to error topic .

Is there any way to do so ??

Re: PublishKafkaProcessor , request is not going to failure relation when kafka broker value is wrong

ckumar — Thu, 28 Dec 2023 13:23:53 GMT

Looking at the thread, Ans is No, that's not the way NiFi Processor is meant to handle relationships, The Client Processor has to initialize /start properly with the given configuration then only it can try to write and then based on the write results flow files can be routed to failure or success, Here Processor can't even start properly.