Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to add component SPARK_JOBHISTORYSERVER to SPARK service, which already installed

Solved Go to solution

How to add component SPARK_JOBHISTORYSERVER to SPARK service, which already installed

New Contributor

My original goal is move the component to another node.

So I just did as Panwar said in https://community.hortonworks.com/questions/4272/process-for-moving-hdp-services-manually.html.I did not call API in order like Panwar tips:

SPARK>:8080/api/v1/clusters/clustername/services/SPARK
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Component"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://:8080/api/v1/clusters/clustername/services/SPARK>:8080/api/v1/clusters/clustername/hosts/<old...
curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://:8080/api/v1/clusters/clustername/services/SPARK>:8080/api/v1/clusters/clustername/services/S... 

I delete the component when it's still running:

curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://namenode01.will.com:8080/api/v1/clusters/datacenter/services/SPARK/components/SPARK_JOBHISTOR...

then, the component disappeared from the stack. I cannot find it from http://namenode01.will.com:8080/api/v1/clusters/datacenter/services/SPARK/components now:

{
  "href" : "http://namenode01.will.com:8080/api/v1/clusters/datacenter/services/SPARK/components",
  "items" : [
    {
      "href" : "http://namenode01.will.com:8080/api/v1/clusters/datacenter/services/SPARK/components/SPARK_CLIENT",
      "ServiceComponentInfo" : {
        "cluster_name" : "datacenter",
        "component_name" : "SPARK_CLIENT",
        "service_name" : "SPARK"
      }
    },
    {
      "href" : "http://namenode01.will.com:8080/api/v1/clusters/datacenter/services/SPARK/components/SPARK_THRIFTSERVER",
      "ServiceComponentInfo" : {
        "cluster_name" : "datacenter",
        "component_name" : "SPARK_THRIFTSERVER",
        "service_name" : "SPARK"
      }
    }
  ]
}

It also disappeared from the UI.

So, anybody help me, please?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to add component SPARK_JOBHISTORYSERVER to SPARK service, which already installed

Here are steps to move SHS from one node to another. Replace admin:admpw with your user-name:password. Run on the Ambari server, or replace localhost with your Ambari server FQDN. Replace "mycluster" with your cluster name. Replace host2.example.com with your target node FQDN.

  • Delete SHS component (it looks like you already did this so you can skip this step)
$ curl -i -H "X-Requested-By:ambari" -u admin:admpw -X DELETE http://localhost:8080/api/v1/clusters/mycluster/services/SPARK/components/SPARK_JOBHISTORYSERVER 
  • Add SHS again
$ curl -i -H "X-Requested-By:ambari" -u admin:admpw -X POST http://localhost:8080/api/v1/clusters/mycluster/services/SPARK/components/SPARK_JOBHISTORYSERVER 
  • Install host component on the new node: host2.example.com
$ curl -i -u admin:admpw -H "X-Requested-By:ambari" -i -X POST -d '{"host_components" : [{"HostRoles":{"component_name":"SPARK_JOBHISTORYSERVER"}}] }' 
  http://localhost:8080/api/v1/clusters/mycluster/hosts?Hosts/host_name=host2.example.com 
  • In Ambari, Hosts --> host2.example.com, click on Spark History Server, and then "Reinstall" button
  • In Ambari-->Spark, start Spark service, and run Spark service check
3 REPLIES 3

Re: How to add component SPARK_JOBHISTORYSERVER to SPARK service, which already installed

New Contributor

another question: I cannot find SPARK_THRIFTSERVER in /var/lib/ambari-server/resources/common-services/SPARK/1.2.0.2.2/metainfo.xml. Why does it exists in SPARK service?

Re: How to add component SPARK_JOBHISTORYSERVER to SPARK service, which already installed

Expert Contributor
@will chen

you can add spark johistoryserver back by using ambari-web UI:

  1. Go to the host details page. This can be done by clicking on the hosts tab and clicking on the host where you want to install spark jobhistoryserver
  2. Click on Add button. It should show list of components that can be added to the host.
  3. Click on spark jobhistoryserver. please see the attached image add-component.png

If for any reason you are not able to add it from UI or you want to use ambari-server REST APIs to do so then article at link has a section "Step 5 - Create host components" which can be used to add host components.

another question: I cannot find SPARK_THRIFTSERVER in /var/lib/ambari-server/resources/common-services/SPARK/1.2.0.2.2/metainfo.xml. Why does it exists in SPARK service?

Thriftserver support for spark was added first in spark 1.4.1 definition in ambari (link). It will be defined at /var/lib/ambari-server/resources/common-services/SPARK/1.4.1/metainfo.xml.

Re: How to add component SPARK_JOBHISTORYSERVER to SPARK service, which already installed

Here are steps to move SHS from one node to another. Replace admin:admpw with your user-name:password. Run on the Ambari server, or replace localhost with your Ambari server FQDN. Replace "mycluster" with your cluster name. Replace host2.example.com with your target node FQDN.

  • Delete SHS component (it looks like you already did this so you can skip this step)
$ curl -i -H "X-Requested-By:ambari" -u admin:admpw -X DELETE http://localhost:8080/api/v1/clusters/mycluster/services/SPARK/components/SPARK_JOBHISTORYSERVER 
  • Add SHS again
$ curl -i -H "X-Requested-By:ambari" -u admin:admpw -X POST http://localhost:8080/api/v1/clusters/mycluster/services/SPARK/components/SPARK_JOBHISTORYSERVER 
  • Install host component on the new node: host2.example.com
$ curl -i -u admin:admpw -H "X-Requested-By:ambari" -i -X POST -d '{"host_components" : [{"HostRoles":{"component_name":"SPARK_JOBHISTORYSERVER"}}] }' 
  http://localhost:8080/api/v1/clusters/mycluster/hosts?Hosts/host_name=host2.example.com 
  • In Ambari, Hosts --> host2.example.com, click on Spark History Server, and then "Reinstall" button
  • In Ambari-->Spark, start Spark service, and run Spark service check
Don't have an account?
Coming from Hortonworks? Activate your account here