Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

JobHistory server rolegroup cannot be added to HostTemplate

JobHistory server rolegroup cannot be added to HostTemplate

New Contributor

Hi,

I try to create a cluster via cloudera manager api in the following way:
1. create a cluster
2. add hosts to the cluster
3. create HDFS / YARN / ZOOKEEPER services using rolegroups
4. create HostTemplates for master and slave nodes.

My problem is that when I create the master HostTemplate there is no JobHistory role listed among YARN roles. Because of it after I assign the templates to the hosts, jobhistory server will be missing.

Can somebody tell me what's the reason of it, please? I use CDH 5.1.2 with cloudera manager.

My other problem is that if I add the JobHistory server manually via the admin console ui and start the cluster, the example mapreduce jobs cannot be executed. I think the reason is that the value of the yarn.nodemanager.aux-services is empty in yarn-site.xml but the appropriate cloudera manager property set to true (mapreduce_aux_service).

Can somebody explain me the reason of it?

Thanks and regards,
Feri

 

3 REPLIES 3

Re: JobHistory server rolegroup cannot be added to HostTemplate

Expert Contributor

Any progress on that matter? In CM version 5.5.1 it is still not possible to add the JobHistory Server to a Host template.

 

Thanks, Benjamin

 

jobhjistory.PNG

Re: JobHistory server rolegroup cannot be added to HostTemplate

CM currently does not allow singleton roles (roles that can only be assigned to a single host) in templates, though we're looking to relax this in the future.

The main purpose of host templates is to stamp out new copies of a edge node or worker node, so singleton roles aren't expected to be very useful in host templates.

I suspect that your job history server is having issues because you didn't call the appropriate initialization commands. Have you tried the first run API, which you'd normally run after assigning all roles?

There's a cluster-level one that's handy if you're initializing the entire cluster, and service-level one that's handy if you're initializing just a single service:
http://cloudera.github.io/cm_api/apidocs/v11/path__clusters_-clusterName-_commands_firstRun.html
/clusters/{clusterName}/services/{serviceName}/commands/firstRun

Thanks,
Darren

Re: JobHistory server rolegroup cannot be added to HostTemplate

Expert Contributor

I'm currently using templates to preserve the role assignment of the current cluster before removing all the nodes. Afterwards I'm rebuilding the cluster on a set of new nodes using those templates. Thus, adding the JHS role to a template would be helpful.

 

btw: Do you have any knowledge ressources regarding cluster migration to new nodes? The documentation only provides sparse information on that matter (relevant paths of HDFS to migrate are missing for instance).