Problem in spark jobs

[#1]

I am using spark to insert data into hive from avro files.On successful completion i am sending email using spark only.In spark submit command when i am running the jar in the client mode the email is triggering, and when running in cluster mode the email do not trigger.

What is the possible solution for this?

[#2]

In client mode your jar is running on edge node or local machine which will have smtp connectivity. In cluster mode, any of the data nodes could run your jar, so you will need to check connectivity to smtp from all nodes.

[#3 · Accepted solution]

@Aditya Jadia

This could probably be related to firewall/communication issue from within cluster worker nodes to email server. To find out exactly what went wrong you should fetch the application logs. yarn logs -applicationId <appId> - Also test email server from within cluster worker nodes with different tool (like telnet) to check connectivity.

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

[#4]

Thanks.Logs helped me