PutEmail frozen

[#1]

Hello,

I have a simple process getting data from sql server and send them by email. Short emails, nothing heavy and not that many during the day. But few days ago, it suddenly started to be frozen and it happened again today. It does not show any error and I do not see any error in logs either.

I checked this thread, maybe it's the same, but there is no real solution and in my logs I can't see anything with SMTP or that it's "waiting", so would like to ask you for help or to point me to the right direction.

Maybe I'm not checking everything...

Thanks

Rgds

Ludvik

[#2]

@Luwi,

Set you processor on Debug and see if any new lines get written. Are you certain that the port is correct and you have a firewall access from NiFi to that stmp server via that port? Besides that, are you certain that SMTP STARTTLS should be on true? I am using it on false and had not issue with it. In addition to this, try sending an text/plain email from a generateFlowFile processor to test the functionality. Maybe it has something to do with generating the html layout for your email 

[#3]

Hi, thanks a lot for your reply. This setup worked for month with no issues until last Friday, since then it got stuck twice. I changed port to 25 yesterday and no problems so far, so hopefully this will solve it. Thanks for pointing me to SMTP Starttls, in case of troubles I will definitely try to set it to false.

I will monitor it closely and if no issued within next month, I will accept you comment as solution, because your suggested changing port too. Thanks again, I will drop a comment later so this topic is updated.

[#4]

So, there is some development on this. After several hours of working without issues on changed port 25 it got stuck again. So I changed STARTTLS to false and got error below. Then changed it back to "true" and got another error related to the port, se below as well.

Emails has always subject like this: (NiFi) FP05-5213923, 88918, User Name, 101 City

Body is always one line: Nové číslo faktury: 5213923

[#5]

@Luwi 
Your screen shot while showing no tasks completed in the past 5 minutes, does show that your PutEmail processor has an active thread (denoted by the (1) in the upper right corer).  In a later screen shot you shared I see that you have a terminated active thread (denoted by the 0(1) in the upper right corner).  Terminating does not kill the thread in Java (only a restart of the entire JVM can kill off an active thread).  All "terminate" does is isolate the terminated thread and release the FlowFile tied to that thread back to the inbound connection queue.  A terminated thread will hang around until it finally competes (provided it is not hung indefinitely).

In scenarios like this where you have a very long running thread / hung thread, you'll want to collect a series of thread dumps for your NiFi and look for the email thread to see if it making any progress between thread dumps (indicates a potentially long running thread) or every thread dump is the same (indicates a potentially hung thread).  Then you'll want to start your investigation around that thread state to figure out why it is not progressing which may require you looking at the configured email server.  Perhaps it has some kind of quota on max emails over a period of time and starts blocking???  If you have not restarted your NiFi yet, you may also be able to take a look at the "terminated" thread in the thread dump to see what it's dump shows.

As far as the other thread you referenced related to the hung thread, the following Jira was created to add ability to include dynamic smtp mail properties: 
https://issues.apache.org/jira/browse/NIFI-9758

It was addressed as of Apache NiFi 1.17 and your screenshot indicates you are running Apache NiFi 1.16, so you'll need to upgrade to get this new processor functionality. 
You can find a list of mail.* properties and their description here:
https://javaee.github.io/javamail/docs/api/index.html?com/sun/mail/smtp/package-summary.html

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

[#6]

Hi Matt,

thanks for your reply and suggestions, appreciate it. Server where NiFi is running is restarted every night. I had to terminate it to release processor, so it can work again. I set the processor on Debug as suggested by @cotopaul and will see tomorrow more details I believe.

Yes, there is probably something with exchange communication, but it should not be some kind of quota on max emails, as when I terminate it and run again, it works for several hour again. Also, it rarely process more then 10 emails at once and we check this one.

Thanks for info about dynamic smtp mail properties, I will go through this.

Will inform tomorrow on my progress. 🙂

Thanks a lot!

Ludvik

[#7 · Accepted solution]

@Luwi 
I agree that it is likely not some kinda of quota and rather a connection interruption and since the default timeout in the smtp client is indefinite, the PutEmail processor will never timeout the bad connection and then execute a new thread to establish a new connection.  The Jira (https://issues.apache.org/jira/browse/NIFI-9758) I mentioned in previous response provides the new capability to set smtp timeouts to resolve this issue.  You'll need to upgrade to Apache NiFi 1.17 or newer to get this processor improvement.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt