Spark action always submit to 0.0.0.0:8032

[#1]

I'm trying to submit a spark action using Oozie with master=yarn-cluster and mode=cluster but the job hangs trying to submit to RM 0.0.0.0:8032

In the YARN logs of the application there are the the following 2 lines:

org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at hdp24node00/192.168.178.30:8050
org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
...
...
org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032

It first try to connect to the right resource manager, but then defaults to 0.0.0.0:8032 until it fails.

yarn.resourcemanager.address is set to hdp24node00:8050 and job-tracker property is also set to hdp24node00:8050.

Where is the problem?

[#2]

I am running HDP 2.4.2 and here is the stack trace from yarn node manager logs

2016-06-30 12:17:23,376 INFO localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(711)) - Created localizer for container_e11_1467282483822_0010_02_000001

2016-06-30 12:17:23,378 INFO localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(866)) - Failed to download resource { { file:/hadoop/yarn/local/filecache/117/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0-258.jar, 1467269880000, FILE, null },pending,[(container_e11_1467282483822_0010_02_000001)],22592100725522,DOWNLOADING}

java.io.FileNotFoundException: File file:/hadoop/yarn/local/filecache/117/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0-258.jar does not exist

at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:609)

at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:822)

at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:599)

at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)

at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:253)

at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:358)

at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:62)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

[#3]

We've three nodes cluster of HDP 2.4 stack. We are trying to run spark jobs using oozie workflow, did all the workaround mentioned in this post as well as posted in other Hortonworks community. But still we are getting error:

"Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, Call From ip-172-63-10-15.eu-west-1.compute.internal/172.63.10.15 to 0.0.0.0:8032 failed on co nnection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused java.net.ConnectException: Call From ip-172-63-10-15.eu-west-1.compute.internal/172.63.10.15 to 0.0.0.0:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused"

Any suggestion, what we are missing and what settings needs to done?