Capture Error messages from NiFi bulletins

[#1]

I would like to capture error messages from bulletin in NiFi and store them in Postgres for audit purposes. I tried with InvokeHTTP processor and NiFi Rest-API end points available but couldn't fetch any error message. Am I missing any step here? Suggest how to get this done.

[#2]

Have you confirmed that error messages exist on the bulletin?

Also...

NiFi logs (errors) are stored in ambari solr infra instance. You can grab all errors/warnings from ambari-infra and push to postgres.

[#3]

@Sunile Manjee

Yes Sunile I have created a scenario to get an error and I see the error bulletin. PFB.

When I start the above invoke-http processor I get an HTML file without error log, I couldn't infer anything from it. PFB.

And I don't see the error in Ambari solr infra webpage which I see in bulletin OR in /var/log/ambari-infra-solr . Can you please suggest how to grab errors/warnings from ambar-infra?

Thanks.

[#4]

Hi @spdvnz, I'm interested in this too; when I asked around earlier here in HCC about error capturing, one recommendation that was given was to grab the error data from nifi-app.log using the "tail file" approach. We've not implemented it yet, thought I share it with you.

[#5]

Yes Raj, I went through all related questions here in HCC. I implemented tail file with nifi-app.log file but we need to process each row from this log as a flowfile and route it based on error/warn/exception keywords in it. And parse it for pushing it to each column of the Postgres table. Moreover in Ambari enabled NiFi nifi-app.log doesn't contain error messages but only heartbeat log, I couldn't figure out which log file have the errors as of now.

To avoid the above-mentioned way I wanted to make use of Rest-API using which we can pick up the bulletins. This seems much easier way.

[#6]

Hi @spdvnz @Raj B , Can you let me know how were you able to grab the failures eventually? I've tried storing the bulletin board messages in hdfs using the REST API but the Json generated from there is very detailed and would require a lot of work before I can use it for monitoring purpose. Would like to know how you guys did it?
Here is the link to the actual question
https://community.hortonworks.com/questions/171150/monitoring-nifi-flow-file-failures-success-1.html