Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Expert Contributor

Hello I once had this problem on oozie:

.
.
.
2016-01-21 14:20:05,033 INFO [main] org.apache.spark.SparkContext: Added JAR hdfs://sandbox.hortonworks.com:8020/user/spark/MlcTransform/lib/mlc-transform-0.3-SNAPSHOT.jar at hdfs://sandbox.hortonworks.com:8020/user/spark/MlcTransform/lib/mlc-transform-0.3-SNAPSHOT.jar with timestamp 1453386005032
2016-01-21 14:20:05,148 WARN [main] org.apache.spark.metrics.MetricsSystem: Using default name DAGScheduler for source because spark.app.id is not set.
2016-01-21 14:20:05,258 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
2016-01-21 14:20:11,274 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:12,276 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:13,277 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:14,278 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:15,279 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:16,280 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:17,282 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:18,284 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:19,285 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-01-21 14:20:20,286 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
.
.
.

Once I change jobTracker=sandbox.hortonworks.com:8032 and in ambari -> yarn ->yarn.resourcemanager.address to the same then I was able to run oozie job with spark actions and master = yarn-client.

However if I dont want to use port = 8032 (the default on the other 2 are 8050 I think), how do I change this on oozie?

21 REPLIES 21
Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Mentor

@David Tam

please refer to the following list of configured ports for YARN. In your job.properties for jobtracker, replace port 8032 with 8050.

so in essence, your job.properties will look like so:

jobTracker=sandbox.hortonworks.com:8050

here's an example.

Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Expert Contributor

@Artem Ervits - yes I did that - the log is in fact what I have got when I have jobTracker=sandbox.hortonworks.com:8050 and in ambari -> yarn ->yarn.resourcemanager.address to the same 8050.

It seems that oozie only works on 8032?

Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Mentor

@David Tam you also have this in your log spark.app.id is not set

Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Expert Contributor

thanks! I will set it now.

Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

I'm having the same issue. The jobTracker port in my workflow/job.properties is set to 8050 (to match the yarn setting) and I can see in the oozie UI (click on job > action > action configuration) that 8050 is being used:

...
<job-tracker>mydomain:8050</job-tracker>
...

But when I drill down into the hadoop job history logs I see the error:

Call From mydomain to 0.0.0.0:8032 failed on connection exception: java.net.ConnectException: Connection refused

Where is it pulling 8032 from? Why does it not use the port configured in the job.properties?

"The workflow only started to work once I changed both of these to 8032"

I'd rather not do this, is there a way to get it to respect the port in the job.properties?

Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Mentor

@Breandán Mac Parland please open this as a new question and provide your workflow as well as job.properties

Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Mentor

@David Tam the example I linked you works on Sandbox. Did you make any modifications to Yarn?

Highlighted

Re: How to change ResourceManager port on oozie (i.e. override the default 8032 to e.g. 8050)?

Expert Contributor

@Artem Ervits : yes I did.

The original yarn ->yarn.resourcemanager.address in ambari was set to 8050 on the sandbox when I got it.

I then change the jobtracker in workflow.xml to 8050 (to match yarn), and I got the log in the question above.

The workflow only started to work once I changed both of these to 8032

Don't have an account?
Coming from Hortonworks? Activate your account here