Member since
07-18-2016
12
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
41643 | 08-31-2018 04:21 AM |
11-21-2019
10:57 PM
Connection reset simply means that a TCP RST was received. This happens when your peer receives data that it can't process, and there can be various reasons for that. The simplest is when you close the socket, and then write more data on the output stream. By closing the socket, you told your peer that you are done talking, and it can forget about your connection. When you send more data on that stream anyway, the peer rejects it with an RST to let you know it isn't listening.
... View more
08-31-2018
09:01 AM
FLUME-3027 has been backported to CDH5.11.0 and above, so if you are able to upgrade, it would prevent the issue of offsets bouncing back and forward. One thing you may want to consider, if you are getting rebalances, it may be because it is taking too long to deliver by your sink, before polling kafka again. You may want to consider lowering your sink batch size in order to deliver and ack the messages in a timely fashion. Additionally, if you upgrade to CDH5.14 or higher, the flume kafka client is 0.10.2, and you would be able to set max.poll.records to match the batchSize you are using for the flume sink. Additionally, you could increase the max.poll.interval.ms, which is decoupled from the session.timeout.ms in 0.10.0 and above. This would prevent the rebalancing from occurring since the client would still heartbeat without having to do a poll to pull more records before session.timeout.ms expires. -pd
... View more
04-10-2017
03:33 AM
Hi, We are also getting similar error like below: WARN Auto offset commit failed for group console-consumer-26249: Offset commit failed with a retriable exception. You should retry committing offsets. (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) We have three node cluster. If we kill one of the Kafka node then remaining two nodes hang and continuely gave above message without consume any data. If we bring up the down node again then all are trying to consume data without above warning/exception message. we are using kafka 0.10.1.1 version and linux machine. we are tried below consumer properties. But no luck. enble.auto.commit = true
auto.commit.interval.ms = 1000 zhuangmz : we can't restart the cluster in production.It is not acceptable solutions at production environment. Any specific properties to resolve this group coordination. Thanks in Adv. Thanks Yarra
... View more