Created 12-26-2023 09:05 AM
In my flow iam publishing data to kafka topic , if there is failure , that request should go to failure relation where iam publishing that data to error topic , but in my case when there is not connectivity with kafka or kafka broker value is not correct request is getting queue in PublishKafka processor not moving to failure relation .
This queue will effect my other requests any even my error handling logic .Please suggest some solution
Attaching the image of error and my nifi flow-
Created 12-27-2023 02:01 PM
PublishKafka must have an active connection to Kafka before it even attempts to send a FlowFile which mean it never even gets into the block of code that sends it and routes it to "success" or "failure" accordingly. Making sure your Kafka cluster is up and running should be the focus if this is what you're experiencing. My guess if this "error" topic you have is on the same Kafka cluster then, even if PublishKafka was able to route a FlowFile to "failure" when it's unable to connect to Kafka, it wouldn't work anyways.
Created 12-27-2023 10:46 PM
My "error" topic is on different Kafka cluster . So i wanted my data which is failing due to connection error should move to failure and then to error topic .
Is there any way to do so ??
Created 12-28-2023 05:23 AM
Looking at the thread, Ans is No, that's not the way NiFi Processor is meant to handle relationships, The Client Processor has to initialize /start properly with the given configuration then only it can try to write and then based on the write results flow files can be routed to failure or success, Here Processor can't even start properly.