Created on 06-19-2014 07:23 AM - edited 09-16-2022 02:00 AM
I originally installed YARN, then MAPREDUCE on my CM5/CDH5 cluster on CentOS 6. I then removed YARN the best way I know how, but removing all nodes from YARN, then deleting YARN from my cluster.
Now, when I run a MR job it is still trying to connect to the YARN resource manager:
cloudera mapreduce INFO client.RMProxy: Connecting to ResourceManager at masternode.dns.name/999.999.999.999:8032
I removed and re-added MAPREDUCE, deploying the client configuration at each step. Clearly there are vestigal remains of YARN. How can I "un-ring" the YARN bell?
Created 06-19-2014 02:15 PM
Okay, I found the problem. It was in the alternatives system (why did I not look there before?!?!?!?).
Apparently when I installed YARN it added a path to hadoop-conf of /etc/hadoop/conf.cloudera.yarn and made it a HIGHER PRIORITY than the /etc/hadoop/conf.cloudera.mapreduce that I need. Simply removing the path worked:
alternatives --remove hadoop-conf /etc/hadoop/conf.cloudera.yarn
Created 06-19-2014 10:55 AM
Created 06-19-2014 12:38 PM
Thank you for the tip, but no joy. 😞
In fact, simply running "mapred queue -list" from the command line attempts to connect to the YARN resource manager!
I even went so far as to delete and recreate the cluster. I cannot get rid of YARN, no way, no how . . .
Created 06-19-2014 02:15 PM
Okay, I found the problem. It was in the alternatives system (why did I not look there before?!?!?!?).
Apparently when I installed YARN it added a path to hadoop-conf of /etc/hadoop/conf.cloudera.yarn and made it a HIGHER PRIORITY than the /etc/hadoop/conf.cloudera.mapreduce that I need. Simply removing the path worked:
alternatives --remove hadoop-conf /etc/hadoop/conf.cloudera.yarn