Reply
Highlighted
Explorer
Posts: 18
Registered: ‎01-30-2014
Accepted Solution

Running MR on CDH 5 cluster tries to connect to YARN resource manager

I originally installed YARN, then MAPREDUCE on my CM5/CDH5 cluster on CentOS 6.  I then removed YARN the best way I know how, but removing all nodes from YARN, then deleting YARN from my cluster.

 

Now, when I run a MR job it is still trying to connect to the YARN resource manager:

 

cloudera mapreduce INFO client.RMProxy: Connecting to ResourceManager at masternode.dns.name/999.999.999.999:8032

 

I removed and re-added MAPREDUCE, deploying the client configuration at each step.  Clearly there are vestigal remains of YARN.  How can I "un-ring" the YARN bell?

Cloudera Employee
Posts: 62
Registered: ‎07-29-2013

Re: Running MR on CDH 5 cluster tries to connect to YARN resource manager

Hi Bogolese,
I am definitely not a matter of authority when it comes to this but
perhaps, you need to re-deploy MR1 client configuration. Have you tried
that?

Click on the MR1 service, then go to Actions->Deploy Client Configuration
on the top right hand side of the screen.

Explorer
Posts: 18
Registered: ‎01-30-2014

Re: Running MR on CDH 5 cluster tries to connect to YARN resource manager

Thank you for the tip, but no joy.  :-(

 

In fact, simply running "mapred queue -list" from the command line attempts to connect to the YARN resource manager!

 

I even went so far as to delete and recreate the cluster.  I cannot get rid of YARN, no way, no how . . .

Explorer
Posts: 18
Registered: ‎01-30-2014

Re: Running MR on CDH 5 cluster tries to connect to YARN resource manager

Okay, I found the problem.  It was in the alternatives system (why did I not look there before?!?!?!?).

 

Apparently when I installed YARN it added a path to hadoop-conf of /etc/hadoop/conf.cloudera.yarn and made it a HIGHER PRIORITY than the /etc/hadoop/conf.cloudera.mapreduce that I need.  Simply removing the path worked:

 

alternatives --remove hadoop-conf /etc/hadoop/conf.cloudera.yarn

Announcements