Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Blueprint to start and stop the ambari services

Blueprint to start and stop the ambari services

New Contributor

Hi all

I am new in Hodooping and work now with Sandbox version 2.5.

I implement new service for Ambari and use docker-compose to start Hortonworks Sandbox with newly installation of my service.

I mentioned, that after restart my service doesn't start.

I tried all suggestions from https://community.hortonworks.com/articles/71748/how-do-i-enable-automatic-restart-recovery-of-serv.... except 'Blueprint'

The API

curl -u admin: -H "X-Requested-By: ambari"-X PUT 'http://localhost:8080/api/v1/clusters/ /components?ServiceComponentInfo/component_name.in(HBASE_REGIONSERVER)'-d '{"ServiceComponentInfo" :                          {"recovery_enabled":"true"}}'

returns

{  "status": 500,
  "message": "Server Error"}

I tried to update Ambari configuration using

 update servicecomponentdesiredstate set recovery_enabled = 1 where component_name in ('my_comp_name');

but this didn't help also.

Could someone share knowledge how I can fix this issue with Blueprint (or other ideas) ?

Thanks in advance!

10 REPLIES 10
Highlighted

Re: Blueprint to start and stop the ambari services

Rising Star

Hi @Yurii Tymchii, Are you trying to create a new cluster with specific recovery/restart settings?

If so, the following wiki doc link might be helpful:

https://cwiki.apache.org/confluence/display/AMBARI/Recovery%3A+auto+start+components#Recovery:autost...

It is possible to configure the "recovery_enabled" settings via a Blueprint deployment. I believe that this will work on a per-service or per-component basis, depending upon the needs of your cluster deployment.

The "REST API" section of the document mentioned above might also be useful, if you are trying to update the recovery settings on an existing cluster using the Ambari REST API.

Re: Blueprint to start and stop the ambari services

Expert Contributor

@Yurii Tymchii

In addition to what @rnettleton suggested, please also share detailed error that will be logged when the API call resulting in 500 server error is made.

ambari-server log is located at /var/log/ambari-server/ambari-server.log location on the host.

This will help us to understand the cause of the failure.

Re: Blueprint to start and stop the ambari services

Here is what I tried on a deployed cluster and it worked: API to get the status of recovery:

curl -u admin:admin -H "X-Requested-By: ambari" -X GET 'http://localhost:8080/api/v1/clusters/c1/services/HDFS/components/DATANODE'

API to enable recovery of datanode

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"ServiceComponentInfo" : {"recovery_enabled":"true"}}' 'http://localhost:8080/api/v1/clusters/c1/services/HDFS/components/DATANODE'

After that relevant log lines from the agent log:

INFO 2017-02-07 02:13:33,214 RecoveryManager.py:677 - ==> Auto recovery is enabled with maximum 6 in 60 minutes with gap of 5 minutes between and lifetime max being 1024. Enabled components - DATANODE
...
INFO 2017-02-07 02:14:52,438 RecoveryManager.py:255 - DATANODE needs recovery, desired = STARTED, and current = INSTALLED.
...
INFO 2017-02-07 02:15:57,920 Controller.py:397 - Adding recovery command START for component DATANODE
INFO 2017-02-07 02:15:57,920 ActionQueue.py:134 - Adding AUTO_EXECUTION_COMMAND for role DATANODE for service HDFS of cluster c1 to the queue.
INFO 2017-02-07 02:15:57,953 ActionQueue.py:277 - Executing command with id = 1-0, taskId = 1486429429 for role = DATANODE of cluster c1.
INFO 2017-02-07 02:16:02,592 RecoveryManager.py:185 - current status is set to STARTED for DATANODE
INFO 2017-02-07 02:16:02,593 ActionQueue.py:428 - After EXECUTION_COMMAND (START), with taskId=1486429429, current state of DATANODE to STARTED

Re: Blueprint to start and stop the ambari services

Super Mentor

@Yurii Tymchii

Your Curl command syntax is not correct. I see that it has many issues specially missing cluster name. I tried the following and i see it is working as expected:

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT 'http://c6401.ambari.apache.org:8080/api/v1/clusters/ClusterDemo/components?ServiceComponentInfo/component_name.in(HBASE_RGIONSERVER)' -d '{"ServiceComponentInfo" : {"recovery_enabled":"true"}}'

OR

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"ServiceComponentInfo" : {"recovery_enabled":"true"}}' 'http://c6401.ambari.apache.org:8080/api/v1/clusters/ClusterDemo/services/HBASE/components/HBASE_REGIONSERVER'

.

Also while copy pasting the mentioned curl command it might add some special characters in the terminal that are usually not visible. So better to paste the command in a text editor first and then run on your terminal after editing it properly.

.

In both cases you should see the following logging in your "/var/log/ambari-server/ambari-server.log"

07 Feb 2017 02:48:26,545  INFO [ambari-client-thread-494] AbstractResourceProvider:559 - Component: HBASE_REGIONSERVER, oldRecoveryEnabled: false, newRecoveryEnabled true

.

Re: Blueprint to start and stop the ambari services

Super Guru
@Yurii Tymchii

Blueprint is used for automated HDP installation. I think you are looking for Ambari REST APIs to start/stop your custom service. Please correct me if I'm wrong here.

Can you please let us know what error are you getting while starting/stopping custom service from Ambari? You can share logs in this thread.

Re: Blueprint to start and stop the ambari services

New Contributor

Thanks a lot to Jay SenSharma - you was wright - there were some special symbols in my curl :(

Now I got in log next error:

07 Feb 2017 08:19:01,401  WARN [ambari-client-thread-28] AbstractResourceProvider:506 - Can not determine request operation level. Operation level property should be specified for this request.
07 Feb 2017 08:19:01,401  INFO [ambari-client-thread-28] AbstractResourceProvider:516 - Received a updateComponent request: [clusterName=Sandbox, serviceName=CATALOGER, componentName=Sandbox, desiredState=null, recoveryEnabled=null, componentCategory=null]

My JSON to sent is:

'{"ServiceComponent{Info" : {"recovery_enabled" : "true"}}'

Could you please advice how to fix this error

Thanks!

____________________________________________________________________________________

For information - configuration of my service before PUT :

{  "href" : "http://..:8080/api/v1/clusters/Sandbox/services/CATALOGER/components/CATALOG_HIVE",
  "ServiceComponentInfo" : {
    "category" : "SLAVE",
    "cluster_name" : "Sandbox",
    "component_name" : "CATALOG_HIVE",
    "display_name" : "Catalog Hive",
    "init_count" : 0,
    "install_failed_count" : 0,
    "installed_count" : 1,
    "recovery_enabled" : "false",
    "service_name" : "CATALOGER",
    "started_count" : 0,
    "state" : "STARTED",
    "total_count" : 1,
    "unknown_count" : 0
  },
  "host_components" : [
    {
      "href" : "http://..:8080/api/v1/clusters/Sandbox/hosts/sandbox.hortonworks.com/host_components/CATALOG_HIVE",
      "HostRoles" : {
        "cluster_name" : "Sandbox",
        "component_name" : "CATALOG_HIVE",
        "host_name" : "sandbox.hortonworks.com"
      }
    }
  ]
}

Re: Blueprint to start and stop the ambari services

Super Mentor

@Yurii Tymchii

This is because your Custom Service "CATALOGER" (component: CATALOG_HIVE) does not seem to be supporting "recovery_enabled" operation. That's why is is not able to understand the operation.

For Example: In your previous "HBASE_REGIONSERVER" curl command if you will try to post a data '{"ServiceComponentInfo" : {"A":"B"}}' like following which makes no sense for HBASE_REGIONSERVER then you will see the same message in your ambari-server log.

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"ServiceComponentInfo" : {"A":"B"}}' 'http://erie1.example.com:8080/api/v1/clusters/ErieCluster/services/HBASE/components/HBASE_REGIONSERVER'

.

Then you will see the same message :

AbstractResourceProvider:516 - Received a updateComponent request: [clusterName=ErieCluster, serviceName=HBASE, componentName=ErieCluster, desiredState=null, recoveryEnabled=null, componentCategory=null]

.

Re: Blueprint to start and stop the ambari services

Super Mentor

@Yurii Tymchii

Additionally for further research at your end, You should be able to find that out by looking at your Custom Service "CATALOGER" , which should have the "metainfo.xml" :

 /var/lib/ambari-server/resources/common-services/CATALOGER/<version>/metainfo.xml 

.

Do you see any such tag as following in your metainfo.xml file? (unlike other services like HBase, AMS ...etc)

  <recovery_enabled>true</recovery_enabled>

.

In your curl output you might see the "("recovery_enabled" : "false",)" entry but it might not be present in your service definition because If it does not have recovery_enabled field, then get the default from the stack definition. Your custom service should support that operation. Please see:

https://cwiki.apache.org/confluence/display/AMBARI/Recovery%3A+auto+start+components#Recovery:autost...

Re: Blueprint to start and stop the ambari services

New Contributor

Thank you all for participation!

I mentioned that if I have /var/lib/ambari-server/resources/common-services/CATALOGER/<version>/metainfo.xml

before adding service via REST I couldn't install my CATALOGER service :(

The way, how I install my service is here:

https://cwiki.apache.org/confluence/display/AMBARI/Adding+a+New+Service+to+an+Existing+Cluster

But before this I copied necessary xml and python script to /var/lib/ambari-server/resources/stacks/HDP/2.5/services/CATALOGER/

So maybe problem is in the way how I install my service?

Don't have an account?
Coming from Hortonworks? Activate your account here