01-16-2018 12:36 AM - last edited on 01-16-2018 05:41 AM by cjervis
In our application, we have using spark streaming with kafka. We are reading around 300k messages and we have set a batch duration of 5 minutes. We observed that our job runs fines for 150mins and then it started taking more time to process eavh batch which led to increase in schedule delay. Once the issue observed it never comes back from the delay and the delay is getting increased slowly. On checking the logs we found the below INFO. Could you please look into the below message and let us know.
18/01/15 16:36:03 INFO internals.AbstractCoordinator: Marking the coordinator <Kafka_server_name>:9093 (id: 2147483644 rack: null) dead for group spark-executor-<app_name>
18/01/15 16:36:03 INFO internals.AbstractCoordinator: Discovered coordinator <Kafka_server_name>:9093 (id: 2147483644 rack: null) for group spark-executor-<app_name>.
18/01/15 16:36:13 INFO kafka010.KafkaRDD: Computing topic <TOPIC_NAME>, partition 4 offsets 151646612 -> 151817912