Skip to main content
Rohit1997jio
Rohit1997jio
Silver ⭐️⭐️⭐️
December 26, 2023
Question

PublishKafkaProcessor , request is not going to failure relation when kafka broker value is wrong

In my flow iam publishing data to kafka topic , if there is failure , that request should go to failure relation where iam publishing that data to error topic , but in my case when there is not connectivity with kafka or kafka broker value is not correct request is getting queue in PublishKafka processor not moving to failure relation .

This queue will effect my other requests any even my error handling logic .Please suggest some solution

Attaching the image of error and my nifi flow-

publishError.pngpublishError1.png

 

3 replies

joseomjr
joseomjr
Gold ⭐️⭐️
December 27, 2023

PublishKafka must have an active connection to Kafka before it even attempts to send a FlowFile which mean it never even gets into the block of code that sends it and routes it to "success" or "failure" accordingly. Making sure your Kafka cluster is up and running should be the focus if this is what you're experiencing. My guess if this "error" topic you have is on the same Kafka cluster then, even if PublishKafka was able to route a FlowFile to "failure" when it's unable to connect to Kafka, it wouldn't work anyways.

Translate
    Rohit1997jio
    Rohit1997jioAuthor
    Silver ⭐️⭐️⭐️
    December 28, 2023

    My  "error" topic  is on different Kafka cluster . So i wanted my data which is failing due to connection error should move to failure and then to error topic .

    Is there any way to do so ?? 

    Translate
    ckumar
    Platinum ⭐️
    ckumar
    Platinum ⭐️
    December 28, 2023

    Looking at the thread, Ans is No, that's not the way NiFi Processor is meant to handle relationships, The Client Processor has to initialize /start properly with the given configuration then only it can try to write and then based on the write results flow files can be routed to failure or success, Here Processor can't even start properly. 

     

    Translate
    Terms & ConditionsAccessibility statement