- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
is it possible to stop/start kafka service each kafka node separated not in parallel
- Labels:
-
Apache Ambari
-
Apache Kafka
Created on
‎02-27-2020
08:30 AM
- last edited on
‎02-27-2020
12:07 PM
by
ask_bill_brooks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi all
in our cluster HDP - 2.6.4 , we have kafka cluster with 11 nodes ( 11 machines )
restart the kafka service , from AMBARI REST API , will stop/start is brutal
when we do it from rest API then its cause corrupted indexes files in kafka topic partitions
what we search is , ambari rest API that do the restart in parallel
for example
first stop kafka01
then
start kafka01
wait until alert gone (from ambari)
then stop kafka02 , and start kafka02
etc....
is this process could be done by ambari rest API?
Created ‎02-27-2020 07:40 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In addition to my previous comment:
Ambari provides option to "Rolling Restart Kafka Brokers" (one by one). In the ambari UI when you click on
Ambari UI --> Kafka --> Service Actions (drop down) --> "Restart Kafka Brokers"
Then it basically shows the Rolling restart Settings like. You can decide how much time in your env one Kafka Broker full restart taks and then after how much time do you want other Kafka broker restart will be scheduled by Ambari.
It can also be achieved using the API call as following:
# curl -iskH "X-Requested-By: ambari" -u admin:admin -X POST -d '[{"RequestSchedule":{"batch":[{"requests":[{"order_id":1,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.1.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode2.example.com"}]}},{"order_id":2,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.2.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode3.example.com"}]}},{"order_id":3,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.3.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode5.example.com"}]}}]},{"batch_settings":{"batch_separation_in_seconds":"121","task_failure_tolerance":1}}]}}]' ttp://testnode1.example.com:8080/api/v1/clusters/NewCluster/request_schedules
.
Created ‎02-27-2020 07:34 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looks like you have asked a very similar query on the other thread :
Created ‎02-27-2020 07:40 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In addition to my previous comment:
Ambari provides option to "Rolling Restart Kafka Brokers" (one by one). In the ambari UI when you click on
Ambari UI --> Kafka --> Service Actions (drop down) --> "Restart Kafka Brokers"
Then it basically shows the Rolling restart Settings like. You can decide how much time in your env one Kafka Broker full restart taks and then after how much time do you want other Kafka broker restart will be scheduled by Ambari.
It can also be achieved using the API call as following:
# curl -iskH "X-Requested-By: ambari" -u admin:admin -X POST -d '[{"RequestSchedule":{"batch":[{"requests":[{"order_id":1,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.1.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode2.example.com"}]}},{"order_id":2,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.2.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode3.example.com"}]}},{"order_id":3,"type":"POST","uri":"/clusters/NewCluster/requests","RequestBodyInfo":{"RequestInfo":{"context":"_PARSE_.ROLLING-RESTART.KAFKA_BROKER.3.3","command":"RESTART"},"Requests/resource_filters":[{"service_name":"KAFKA","component_name":"KAFKA_BROKER","hosts":"testnode5.example.com"}]}}]},{"batch_settings":{"batch_separation_in_seconds":"121","task_failure_tolerance":1}}]}}]' ttp://testnode1.example.com:8080/api/v1/clusters/NewCluster/request_schedules
.
