Support Questions
Find answers, ask questions, and share your expertise

Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

Highlighted

Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

Here is the sample flow am trying to acheive, am trying to load the data into mysql , i have created two tables am loading the data from one table to other table.

while loading the data to the other table if i see any failures i need to send an email to the Group, along with the log file(nif-app.log).

My flow is ExecuteSql-->SplitAvro-->PutDatabaseRecord-->PutEmailSuccess for Success, PutEmailFailure for Failure.

Here where can i put my TailFile Processor to tail the log file and send to Email. I have tried putting before ExecuteSql it is send an email but it is not sending the log file information. and i have tried to connect TailFile to PutDatabaseRecord it is not allowing me to connect am not sure what is the issue.

would you please help me to resolving it.

13 REPLIES 13
Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

Hi @Nagesh Gollapudi

Better than using nifi logs, you can leverage the S2SBulletingReportingTasks https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-site-to-site-reporting-nar/1....

With reporting task you can send the error to same nifi instance. Once NiFi receives it, it can store it it where ever you want (File, index in Solr, Database, etc) or send it by mail. Basically, you will receive JSON data so you can do what ever you want with this data in nifi by design a flow.

To use it, go to hamburger top-right menu, controller settings, reporting tasks and add a S2SBulletinRT. Configure it to send data to the same cluster like below:

42402-screen-shot-2017-11-01-at-124305-pm.png

Notice the Input Port Name attribute. I called it prov so I need to add an input port prov to NiFi flow. Then, from this input port I'll decide what to do with provenance data (store it somewhere).

Can you test it and see if it suits your needs ?

Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

Hi Abdelkrim Hadjidj,

I have tried to implement the same what you suggested but how can I implement that would you please help me.

I have attached the snaps for your reference.capture.pngcapture1.png

Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

@Nagesh Gollapudi

Have you added the S2SBulletinReportinTask ? what's the issue ? do you have errors ?

Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

Hi Abdelkrim,

I have added the S2SBulletinReportinTask would you please check the snap shots which I have attached, how to implement that I need some help you will get the clear understanding if you check my snaps. please help me to resolve the issue.capture.png, capture1.png


Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

I saw the snapshots but I unfortunately don't see any issue or error. Can you explain what you need ?

Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

Hi Abdelkrim,

See my requirement is whenever I got any failure in the processor I need to send a mail to the group with the nifi-app.log which has error message.

Am just trying to figure out the send a mail when ever then is a failure in the processor, so I have take a simple example here.

My flow is ExecuteSql-->SplitAvro-->PutDatabaseRecord-->PutEmailSuccess for Success, PutEmailFailure for Failure.

Here am doing a sample example, I have two tables in MySQL database (table1, table 2) so to achieve this what am doing it am trying to load the data from table1 to table 2 when it is success am sending the mail with the nifi-app.log saying that records has been inserted.

Second thing is am wanted doing failure and send a email to the group saying that it got failed nifi-app.log.

here the challenge is am trying to send the mails for both success and failures but the log which am sending is not giving the data which is there in nifi-app.log. so where I need to use the Tailfile processor to send the appropriate log file message to the mail. would you please check the snaps and tell me where exactly I need to use it.

I am trying to connect to putdatabaserecord to tailfile is it not accepting because tailfile wont support upstream connection so please help me to achieve this.capture.png,, capture1.png


Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

@Nagesh Gollapudi

To test, start your processor and add a processor that will have error (for instance Put with invalide IP). you should get errors generated and recieved

Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

Hi Abdelkrim,

sorry I was on vacation, I have tried to implement to the solution which you have provided but it didn't worked for me it was showing me the error as below.

"Failed to send Bulletins to destination due to IO Exception: couldn't find port with name "prov" for remote Nifi instance."

I have added the below snippets into nifi.properties file.

# Site to Site properties
nifi.remote.input.host=localhost
nifi.remote.input.secure=false
nifi.remote.input.socket.port=8055
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec

I have added the xml document and error message snap shot could you please look into it and help me to resolve the issue

Thanks in advance.making-the-processor-failed.xml

nifi-error.png

Highlighted

Re: Can we somehow dump the error messgae shown against processor to an email or to a file. In other words, whatever error is being shown in UI of a processor, can we send/dump this in an email

@Nagesh Gollapudi

The input port needs to be at the higher level (root flow) to be found by the S2S protocol, not in the process group "load the load data into mySQL".

You need to add the Input port 'prov' in the root canvas, rename the input port in your process group to something else (prov_internal for instance), and then connect the top level input port (prov) to your process group (load the load data into mysql).

With this, you should be able to receive data from reporting task

Don't have an account?