Created on 12-12-2019 05:50 AM - edited 12-12-2019 05:52 AM
Apache Nifi PutHiveQL How do I extract error message into flowfile attribute when failure occurs?
Created 12-12-2019 11:43 AM
Processors are not coded to expose the exceptions to the FlowFile.
Typically how this handled is by Using the SiteToSIteBulletinReportingTask. Anytime a component throws an exception the processor will display a bulletin in the upper right corner and that same bulletin will be sent to the bulletin board. The same exception along with stack trace (if there is one) is also logged to the nifi-app.log.
The S2SBulletinReportingTasks allows you to capture all produced bulletins and send them via S2S to another NiFi or even back to same NiFi to be processed as FlowFiles for notification and alerting reasons through a dataflow.
Writing exceptions out to FlowFile attributes would be problematic for following reasons:
1. FlowFile attributes exist in NiFi heap memory and exceptions can be large.
2. A FlowFile may pass multiple processor which produce bulletin/exception. Is expectation we overwrite previous exception with new exception? Or create a new FlowFile attribute based on processor uuid for each exception encounter during life of FlowFile? This leads back to point 1 issue.
It is better to handle exceptions as content within their own FlowFiles via the reporting task mentioned above.
Hope this helps,
Matt
Created 12-12-2019 11:43 AM
Processors are not coded to expose the exceptions to the FlowFile.
Typically how this handled is by Using the SiteToSIteBulletinReportingTask. Anytime a component throws an exception the processor will display a bulletin in the upper right corner and that same bulletin will be sent to the bulletin board. The same exception along with stack trace (if there is one) is also logged to the nifi-app.log.
The S2SBulletinReportingTasks allows you to capture all produced bulletins and send them via S2S to another NiFi or even back to same NiFi to be processed as FlowFiles for notification and alerting reasons through a dataflow.
Writing exceptions out to FlowFile attributes would be problematic for following reasons:
1. FlowFile attributes exist in NiFi heap memory and exceptions can be large.
2. A FlowFile may pass multiple processor which produce bulletin/exception. Is expectation we overwrite previous exception with new exception? Or create a new FlowFile attribute based on processor uuid for each exception encounter during life of FlowFile? This leads back to point 1 issue.
It is better to handle exceptions as content within their own FlowFiles via the reporting task mentioned above.
Hope this helps,
Matt