Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

is it possible to stop/start kafka service each kafka node separated not in parallel

avatar

hi all

 

in our cluster HDP - 2.6.4 , we have kafka cluster with 11 nodes ( 11 machines )

 

restart the kafka service , from AMBARI REST API , will stop/start is brutal 

 

when we do it from rest API then its cause corrupted indexes files in kafka topic partitions

 

what we search is , ambari rest API that do the restart in parallel

 

for example

 

first stop kafka01

 

then

 

start kafka01 

 

wait until alert gone (from ambari)

 

then stop kafka02 , and start kafka02

 

etc....

 

 

is this process could be done by ambari rest API?

 

 

Michael-Bronson
1 ACCEPTED SOLUTION

avatar
Master Mentor

@mike_bronson7 

In addition to my previous comment:
Ambari provides option to "Rolling Restart Kafka Brokers" (one by one). In the ambari UI when you click on

Ambari UI --> Kafka --> Service Actions (drop down) --> "Restart Kafka Brokers"

Then it basically shows the Rolling restart Settings like. You can decide how much time in your env one Kafka Broker full restart taks and then after how much time do you want other Kafka broker restart will be scheduled by Ambari.

 

Kafka_Broker_Restart.png


It can also be achieved using the API call as following:

# curl -iskH "X-Requested-By: ambari" -u admin:admin -X POST -d '[{"RequestSchedule":{"batch":[{"requests":[{"order_id":1,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.1.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode2.example.com"}]}},{"order_id":2,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.2.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode3.example.com"}]}},{"order_id":3,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.3.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode5.example.com"}]}}]},{"batch_settings":{"batch_separation_in_seconds":"121","task_failure_tolerance":1}}]}}]' ttp://testnode1.example.com:8080/api/v1/clusters/NewCluster/request_schedules

 

.

View solution in original post

2 REPLIES 2

avatar
Master Mentor

avatar
Master Mentor

@mike_bronson7 

In addition to my previous comment:
Ambari provides option to "Rolling Restart Kafka Brokers" (one by one). In the ambari UI when you click on

Ambari UI --> Kafka --> Service Actions (drop down) --> "Restart Kafka Brokers"

Then it basically shows the Rolling restart Settings like. You can decide how much time in your env one Kafka Broker full restart taks and then after how much time do you want other Kafka broker restart will be scheduled by Ambari.

 

Kafka_Broker_Restart.png


It can also be achieved using the API call as following:

# curl -iskH "X-Requested-By: ambari" -u admin:admin -X POST -d '[{"RequestSchedule":{"batch":[{"requests":[{"order_id":1,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.1.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode2.example.com"}]}},{"order_id":2,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.2.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode3.example.com"}]}},{"order_id":3,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.3.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode5.example.com"}]}}]},{"batch_settings":{"batch_separation_in_seconds":"121","task_failure_tolerance":1}}]}}]' ttp://testnode1.example.com:8080/api/v1/clusters/NewCluster/request_schedules

 

.