question Running MR on CDH 5 cluster tries to connect to YARN resource manager in Archives of Support Questions (Read Only)

Running MR on CDH 5 cluster tries to connect to YARN resource manager

Bogolese — Fri, 16 Sep 2022 09:00:34 GMT

I originally installed YARN, then MAPREDUCE on my CM5/CDH5 cluster on CentOS 6. I then removed YARN the best way I know how, but removing all nodes from YARN, then deleting YARN from my cluster.

Now, when I run a MR job it is still trying to connect to the YARN resource manager:

cloudera mapreduce INFO client.RMProxy: Connecting to ResourceManager at masternode.dns.name/999.999.999.999:8032

I removed and re-added MAPREDUCE, deploying the client configuration at each step. Clearly there are vestigal remains of YARN. How can I "un-ring" the YARN bell?

Re: Running MR on CDH 5 cluster tries to connect to YARN resource manager

mgrover — Thu, 19 Jun 2014 17:55:05 GMT

Hi Bogolese,
I am definitely not a matter of authority when it comes to this but
perhaps, you need to re-deploy MR1 client configuration. Have you tried
that?

Click on the MR1 service, then go to Actions->Deploy Client Configuration
on the top right hand side of the screen.

Re: Running MR on CDH 5 cluster tries to connect to YARN resource manager

Bogolese — Thu, 19 Jun 2014 19:38:53 GMT

Thank you for the tip, but no joy. 😞

In fact, simply running "mapred queue -list" from the command line attempts to connect to the YARN resource manager!

I even went so far as to delete and recreate the cluster. I cannot get rid of YARN, no way, no how . . .

Re: Running MR on CDH 5 cluster tries to connect to YARN resource manager

Bogolese — Thu, 19 Jun 2014 21:15:32 GMT

Okay, I found the problem. It was in the alternatives system (why did I not look there before?!?!?!?).

Apparently when I installed YARN it added a path to hadoop-conf of /etc/hadoop/conf.cloudera.yarn and made it a HIGHER PRIORITY than the /etc/hadoop/conf.cloudera.mapreduce that I need. Simply removing the path worked:

alternatives --remove hadoop-conf /etc/hadoop/conf.cloudera.yarn