Skip to main content
srilakshmi
Silver ⭐️⭐️
Mar 14, 2023 10:30 AM UTC
Solved

Routing based on exception thrown by NIFI processor

Hi ,

 

Currently i am using NIFI to consume and publish data using kafka and i have a requirement to perform retry only for topicAuthorizationException thrown by NIFI processor (publishKafka processor).

 

Like similar to RouteonAttribute, is there any processor or any log attribute through which i can parse or read  this topicAuthorizationException error , so that i can perform retry?

 

when kafka topic does not exist, NIFI throws below exception, the same is seen in nifi-app.log, i need to know how can we read this error in NIFI

 

org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [test_topic]

 

MattWho
MattWhoAnswer
Titanium ⭐️⭐️
Mar 14, 2023 12:56 PM UTC

@srilakshmi 
The PublishKafka and PublishKafkaRecord processors do not write any new attributes to the FlowFile when there is a failure.  It simply logs the failure to the nifi-app.log and routes the FlowFile to the failure relationship.  So on the FlowFile there is no unique error written that can be used for dynamic routing on failure. It could be expensive to write stack traces that come out of Client code to NiFi FlowFiles considering FlowFile attributes/metadata resides in the NiFi heap memory.

This may be a topic you want to raise in Apache NiFi jira as a feature/improvement request on these processors to get feedback from Apache NiFi community committers.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

1 reply

MattWho
Titanium ⭐️⭐️
MattWhoAnswer
Titanium ⭐️⭐️
Mar 14, 2023 12:56 PM UTC

@srilakshmi 
The PublishKafka and PublishKafkaRecord processors do not write any new attributes to the FlowFile when there is a failure.  It simply logs the failure to the nifi-app.log and routes the FlowFile to the failure relationship.  So on the FlowFile there is no unique error written that can be used for dynamic routing on failure. It could be expensive to write stack traces that come out of Client code to NiFi FlowFiles considering FlowFile attributes/metadata resides in the NiFi heap memory.

This may be a topic you want to raise in Apache NiFi jira as a feature/improvement request on these processors to get feedback from Apache NiFi community committers.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

Translate