Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Processor failures: how to get the error on the flowfile "failuire" ?

Solved Go to solution
Highlighted

Processor failures: how to get the error on the flowfile "failuire" ?

Expert Contributor

Hi,

If a processor failed and routed the flowfile to the relationship failure. Is there an attribute "error"? If it's the case for some processor how to know if they have it?
For example, the puthdfs, I don't see anything in the documentation.

doc puthdfs

Is there another way to have the reason of the failure attached to the flowfile,

Thanks,

Michel

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Processor failures: how to get the error on the flowfile "failuire" ?

Expert Contributor

The solution is used the "SiteToSiteBulletinReportingTask" in the reporting task.

It can send all the bulletin to a nifi instance. It can be the same instance the nifi that generate it. It will send it to a specific input port in json, then you will be able to process it. It has all the attributed needed:

Here an example

[{"objectId":"9c8e75e6-eb5a-4a52-9d4a-a3d3b7f0c80f",
"platform":"nifi",
"bulletinId":305,
"bulletinCategory":"Log Message",
"bulletinGroupId":"24a8726b-015f-1000-ffff-ffffae66ea1c",
"bulletinLevel":"ERROR",
"bulletinMessage":"PutHDFS[id=24b463f8-015f-1000-ffff-ffffd09bd856] PutHDFS[id=24b463f8-015f-1000-ffff-ffffd09bd856] failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.; processor will not be scheduled to run for 30 seconds: java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.",
"bulletinNodeId":"ede4721c-30fe-4879-b22e-20bfe602c615",
"bulletinSourceId":"24b463f8-015f-1000-ffff-ffffd09bd856",
"bulletinSourceName":"PutHDFS",
"bulletinSourceType":"PROCESSOR",
"bulletinTimestamp":"2017-10-17T08:16:48.945Z"},

View solution in original post

4 REPLIES 4
Highlighted

Re: Processor failures: how to get the error on the flowfile "failuire" ?

@msumbul

I don't know of any standard error attribute in NiFi. This depends on the processor and usually is not provided.

You can always use UpdateAttribute to add an error type following your needs. If a processor has three possible errors relations, you can have 3 update processors which add an attribute error and populate it with the type of error.

Highlighted

Re: Processor failures: how to get the error on the flowfile "failuire" ?

Expert Contributor

Hi @Abdelkrim Hadjidj,

Thanks for your reply, my objective is to get the error message which can be many thing, hostnotfound, parsing error, connection refuse, etc for the same failure relationship.

Michel

Re: Processor failures: how to get the error on the flowfile "failuire" ?

Explorer

Looking for this info also. Sorry to bump the the thread, but any news on this wish?

Highlighted

Re: Processor failures: how to get the error on the flowfile "failuire" ?

Expert Contributor

The solution is used the "SiteToSiteBulletinReportingTask" in the reporting task.

It can send all the bulletin to a nifi instance. It can be the same instance the nifi that generate it. It will send it to a specific input port in json, then you will be able to process it. It has all the attributed needed:

Here an example

[{"objectId":"9c8e75e6-eb5a-4a52-9d4a-a3d3b7f0c80f",
"platform":"nifi",
"bulletinId":305,
"bulletinCategory":"Log Message",
"bulletinGroupId":"24a8726b-015f-1000-ffff-ffffae66ea1c",
"bulletinLevel":"ERROR",
"bulletinMessage":"PutHDFS[id=24b463f8-015f-1000-ffff-ffffd09bd856] PutHDFS[id=24b463f8-015f-1000-ffff-ffffd09bd856] failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.; processor will not be scheduled to run for 30 seconds: java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.",
"bulletinNodeId":"ede4721c-30fe-4879-b22e-20bfe602c615",
"bulletinSourceId":"24b463f8-015f-1000-ffff-ffffd09bd856",
"bulletinSourceName":"PutHDFS",
"bulletinSourceType":"PROCESSOR",
"bulletinTimestamp":"2017-10-17T08:16:48.945Z"},

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here