Why does Alert Publisher require mail server login & password?

[#1]

Running CDH 5.6.  Thought setting up email alerting would be a straightforward process, but was stymied by the CMS config at "Alerts: Mail Server Username" and "Alerts: Main Server Password".  If Alert Publisher is using standard SMTP, they shouldn't be required, no?

sendmail is running on every node.  Manually running mailx at command line works.  Why can't CMS send mail as the process owner?  Hortonworks/Ambari does not have this issue.

Thanks,

Miles

[#2 · Accepted solution]

Thanks for the additional info. I've seen similar issues before where the fields for "Alerts: Mail Server Username" and "Alerts: Main Server Password". are blank and not null.

Would you be able to check the following for me, open the CM Server URL http://CM-SERVER:7180/api/v11/cm/deployment and search for the following attributes;

...
  "items" : [ {
    "name" : "alert_mailserver_password",
    "value" : ""
  }, {
    "name" : "alert_mailserver_username",
    "value" : ""
  },
... 
  } ]
...

If either attributes are as shown above, it means that Alert publisher is attempting to authenticate with username "" and password "", you will need to set that to NULL.

In the CM UI> Cloudera Management Service> Configuration> 'Switch to the classic layout'> navigate to Alert Publisher Default Group> for "Alerts: Mail Server Username" and "Alerts: Main Server Password"> click: Reset to empty default value

[#3]

Hi Miles,

Thanks for your message. The username password field are optional, users can set a value if their mail server requires authentication.  Can you describe what is the actual error when the values are not set, if possible can you provide your troubleshooting steps that suggested that these fields are required any errors in Alert Publisher or sendmail logs. 

[#4]

Thanks for the reply.  We tried various settings - server FQDN instead of 'localhost', different logins and From field, none worked. 

Alert Publisher log:

2016-05-18 16:23:10,036 INFO com.cloudera.enterprise.alertpublisher.processor.EmailSubjectGenerator.verbose: Generated subject [Cloudera Alert] Test Alert.
2016-05-18 16:23:10,043 ERROR org.apache.camel.processor.DefaultErrorHandler: Failed delivery for exchangeId: ...... Exhausted after delivery attempt: 1 caught: org.springframework.mail.MailAuthenticationException: Authentication failed; nested exception is javax.mail.AuthenticationFailedException: No authentication mechansims supported by both server and client
org.springframework.mail.MailAuthenticationException: Authentication failed; nested exception is javax.mail.AuthenticationFailedException: No authentication mechansims supported by both server and client
at org.springframework.mail.javamail.JavaMailSenderImpl.doSend(JavaMailSenderImpl.java:392)
at org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:340)
at org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:355)
at org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:344)
at org.apache.camel.component.mail.MailProducer.process(MailProducer.java:44)

at org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:50)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:77)
at org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:104)
at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:272)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:98)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:77)
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:99)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:299)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:208)
at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:269)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:77)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:125)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)

at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:109)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:77)
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:77)
at org.apache.camel.component.seda.SedaConsumer.sendToConsumers(SedaConsumer.java:189)
at org.apache.camel.component.seda.SedaConsumer.run(SedaConsumer.java:121)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.mail.AuthenticationFailedException: No authentication mechansims supported by both server and client
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:590)
at javax.mail.Service.connect(Service.java:313)
at org.springframework.mail.javamail.JavaMailSenderImpl.doSend(JavaMailSenderImpl.java:389)
... 31 more

Error messages in /var/log/maillog look like:  "sendmail[12308]: u4ILEPlD012308: localhost [127.0.0.1] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA"

Checked /etc/hosts and /etc/mail - all fairly standard setup, no explicit restrictions.

I can manually send mail from the host with [ date |mailx -s "test" <id@domain> ] from different logins.

What I see may be the cause is that the CMS processes owner is not mapped properly somehow:

111 5755 19346 1 15:46 ? 00:00:17 /usr/java/jdk1.8.0_60/bin/java -server -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Dmgmt.log.file=mgmt-cmf-mgmt-ALERTPUBLISHER-hou706068.int.cggveritas.com.log.out -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Xms1073741824 -Xmx1073741824 -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -cp /var/run/cloudera-scm-agent/process/504-cloudera-mgmt-ALERTPUBLISHER:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/* com.cloudera.enterprise.alertpublisher.AlertPublisher

The UID maps to 'cloudera-scm' locally, but another human user globally in NIS:

/var/log/cloudera-scm-alertpublisher$ id cloudera-scm
uid=111(cloudera-scm) gid=115(cloudera-scm) groups=115(cloudera-scm)

/var/log/cloudera-scm-alertpublisher$ ypcat passwd|grep ':111:'
***:uQeRidWxOqnd2:12118:111:.....

Can this be the reason?  Any workaround other than re-provision the UID?

[#5]

Also, if you leave Mail Server Username/Password blank (default), it will auto-populate to CM 'admin' ID and password the next time around.  Entering a valid Unix ID/password does not help, either (the logs don't tell you whether it even get used).

'cloudera-scm' login is configured nologin, so can't test command-line mail.

[#6 · Accepted solution]

Thanks for the additional info. I've seen similar issues before where the fields for "Alerts: Mail Server Username" and "Alerts: Main Server Password". are blank and not null.

Would you be able to check the following for me, open the CM Server URL http://CM-SERVER:7180/api/v11/cm/deployment and search for the following attributes;

...
  "items" : [ {
    "name" : "alert_mailserver_password",
    "value" : ""
  }, {
    "name" : "alert_mailserver_username",
    "value" : ""
  },
... 
  } ]
...

If either attributes are as shown above, it means that Alert publisher is attempting to authenticate with username "" and password "", you will need to set that to NULL.

In the CM UI> Cloudera Management Service> Configuration> 'Switch to the classic layout'> navigate to Alert Publisher Default Group> for "Alerts: Mail Server Username" and "Alerts: Main Server Password"> click: Reset to empty default value

[#7]

Resetting to default didn't work in the current layout, but works in Classic - thanks!

Do I have to keep classic layout from now on, or can I switch back?

[#8 · Accepted solution]

> Do I have to keep classic layout from now on, or can I switch back?

You can switch back to the new layout. The NULL vs "" is fixed and will be available as soon as CM 5.7.1 is available in our public repo.

[#9]

You can put this option blank..if you are using SMTP, than no need.