04-13-2018 12:07 AM - last edited on 04-13-2018 06:13 AM by cjervis
We are currently using mapreduce Mrv1 framework for job processing and wish to migrate to YARN in CDH5.9.2 cluster.
I came across https://www.cloudera.com/documentation/enterprise/5-3-x/topics/cdh_ig_mapreduce_to_yarn_migrate.html documentation link when I searched. But Its not clear to exactly what steps need to be followed as we will be doing migration on production cluster itself.
The above mentioned document describes the parameters different in mrv1 and yarn .
If you can please help me with the steps need to be followed, that would be great.
04-15-2018 06:38 PM
04-15-2018 11:17 PM
Thank you for inputs. I am usig CM for managing the cluster. The steps mentioned in the link you mentioned https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_yarn_service.html#concept_qgv_... will be same for 5.9 version as well or not?
As part of migration from MRv1 to YARN , I added service to the cluster through add service option.
But one of our important application job was not running after addition of YARN service to the cluster.
04-16-2018 12:09 AM
07-25-2018 04:57 AM
I refered to cloudera this https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_yarn_tuning.htmldocument, all the parameters are specified except yarn.scheduler.minimum-allocation-mb.
And the value for Java Heap Size of NodeManager in Bytes this parameter is too low i.e. 50MB. Other values are good.
Can you please tell me whether this is what causing issue?
07-26-2018 04:24 AM
I have one more query. We are not doing CDH upgrade, just migrating from mapreduce to yarn, so do we have to Import MapReduce Configuration?