Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Auto Start for service when it fails or get stop randomly

avatar

I am looking for new auto start function in which I wan't to start kafka when it fails or stale for sometime and it gets started after some interval. Currently, auto start is giving functionality to start service or reboot which I don't wan't. In my case kafka service get stop's after midnight and there is no one to start it hence it affects the business.any suggestions? I have already tried shell script but due to security issues I couldn't deploy shell scripts.

1 ACCEPTED SOLUTION

avatar
Master Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
5 REPLIES 5

avatar
Master Mentor

@Yee Zee

Ambari Auto start feature is useful when your components (like kafka broker) is abruptly killed/shut down.

Auto start of a component is based on its current state and "desired state".So if you manually stop the services/components then the auto start may not work because the agent compares the current state of these components against the desired state, to determine if these components are to be installed, started, restarted or stopped.

.

You can use some cron job to keep monitoring the status of Kafka Broker ever few minutes and then start them using ambari API call if needed.

Monitoring Kafka Broker State (STARTED / INSTALLED)

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/components/KAFKA_BROKER?fields=host_component...

.

Get the host_name and state from the above ambari API to find the brokers which are down.

.

Starting Kafka Broker if it is down (INSTALLED):

# curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Start Kafka Broker","operation_level":{"level":"HOST_COMPONENT","cluster_name":"$CLUSTER_NAME","host_name":"$KAFKA_BROKER_HOST","service_name":"KAFKA"}},"Body":{"HostRoles":{"state":"STARTED"}}}' "http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/hosts/$KAFKA_BROKER_HOST/host_components/KAFKA_BROKER" 

.

avatar

@Jay Kumar SenSharma Thanks for commenting and explaining, I just wan't to test the scenario to see it working. Is ther any solution to check it's working. I killed the process of kafka manually, but still i didn't see it auto restarting. It would be really helpful if you can guide for alyernative

avatar
Master Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
@Jay Kumar SenSharma

thanks for your answer and details, Actually I need to test auto start.. I tried killing kafka job in shell, but it didn't started via auto -start. any possible scenario to test auto start, when I open ambari for auto start it says:

"Ambari services can be configured to start automatically on system boot." does this statement says that ambari auto start works only when system boot up or any chances that it can also work when service goes down abruptly.

avatar
Master Mentor

@Yee Zee

Please refer to the previously shared detailed steps to test this scenario.

This works for any Abrupt Termination of the component (either due to host reboot or if somehow the process was killed abruptly).

As mentioned earlier that Auto start of a component is based on its current state and "desired state". Which means if you stop a component using Ambari API Or using Ambari UI then you wont see the recovery because in that case the ambari DB will have the correct info.