Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Running MR on CDH 5 cluster tries to connect to YARN resource manager

avatar
Contributor

I originally installed YARN, then MAPREDUCE on my CM5/CDH5 cluster on CentOS 6.  I then removed YARN the best way I know how, but removing all nodes from YARN, then deleting YARN from my cluster.

 

Now, when I run a MR job it is still trying to connect to the YARN resource manager:

 

cloudera mapreduce INFO client.RMProxy: Connecting to ResourceManager at masternode.dns.name/999.999.999.999:8032

 

I removed and re-added MAPREDUCE, deploying the client configuration at each step.  Clearly there are vestigal remains of YARN.  How can I "un-ring" the YARN bell?

1 ACCEPTED SOLUTION

avatar
Contributor

Okay, I found the problem.  It was in the alternatives system (why did I not look there before?!?!?!?).

 

Apparently when I installed YARN it added a path to hadoop-conf of /etc/hadoop/conf.cloudera.yarn and made it a HIGHER PRIORITY than the /etc/hadoop/conf.cloudera.mapreduce that I need.  Simply removing the path worked:

 

alternatives --remove hadoop-conf /etc/hadoop/conf.cloudera.yarn

View solution in original post

3 REPLIES 3

avatar
Expert Contributor
Hi Bogolese,
I am definitely not a matter of authority when it comes to this but
perhaps, you need to re-deploy MR1 client configuration. Have you tried
that?

Click on the MR1 service, then go to Actions->Deploy Client Configuration
on the top right hand side of the screen.

avatar
Contributor

Thank you for the tip, but no joy.  😞

 

In fact, simply running "mapred queue -list" from the command line attempts to connect to the YARN resource manager!

 

I even went so far as to delete and recreate the cluster.  I cannot get rid of YARN, no way, no how . . .

avatar
Contributor

Okay, I found the problem.  It was in the alternatives system (why did I not look there before?!?!?!?).

 

Apparently when I installed YARN it added a path to hadoop-conf of /etc/hadoop/conf.cloudera.yarn and made it a HIGHER PRIORITY than the /etc/hadoop/conf.cloudera.mapreduce that I need.  Simply removing the path worked:

 

alternatives --remove hadoop-conf /etc/hadoop/conf.cloudera.yarn