Member since
03-12-2021
5
Posts
0
Kudos Received
0
Solutions
03-15-2022
06:10 PM
sad news. It was not working. The error message turns to "org.apache.kudu.client.NoLeaderFoundException: Master config (hostnamex:7051,hostnamex:7051,hostnamex:7051)" Are there any reasons to cause this issue? Such as high pressure on KUDU cluster?
... View more
02-28-2022
01:59 AM
thanks for response. the logs are really from client. I checked the KUDU logs for ERROR, but nothing was found. now i have replaced the IP to hostname. hope it works.
... View more
02-28-2022
01:31 AM
transfer the way of connection to hostname? it sounds like choosing a long way instead of the short one. But maybe it works. OK, i'll have a try. Wait for my good news. 🙂
... View more
02-28-2022
12:34 AM
thanks your reply bro, but the ntp sync looks fine because you know CDH cluster has the check for time sync. Is there any configuration suggestion for my kudu cluster? I set 3 masters and 76 kudu tablet servers. The resource for each node is about 48cores and 384G mem and 5T disk space. We have a high read/write working scenario, is it a reason to cause this?
... View more
02-27-2022
11:12 PM
Dear: Recently i met a KUDU issue which was randomly occurred. Below is the details. CDH Version: 5.14.2 KUDU Version: 1.6.0-cdh5.14.2 ERROR Information: 21/11/28 06:56:03 ERROR yarn.ApplicationMaster: User class threw exception: org.apache.kudu.client.NoLeaderFoundException: Master config (10.186.93.6:7051,10.186.93.24:7051,10.186.93.8:7051) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection disconnected,org.apache.kudu.client.RecoverableException: connection disconnected org.apache.kudu.client.NoLeaderFoundException: Master config (10.186.93.6:7051,10.186.93.24:7051,10.186.93.8:7051) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection disconnected,org.apache.kudu.client.RecoverableException: connection disconnected at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:273) at org.apache.kudu.client.ConnectToCluster.access$100(ConnectToCluster.java:49) at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:349) at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:338) at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280) at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315) at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286) at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) at com.stumbleupon.async.Deferred.callback(Deferred.java:1002) ... Caused by: org.apache.kudu.client.RecoverableException: connection disconnected ... 55 more this issue happened when i submitted a SPARK style task to YARN. Most time it is fine, but sometimes the problem come. Maybe several times per day. Everytime I have to re-submit my task to YARN again, luckly the second time try always succeed. Is it a BUG or something else? Is there any way to fix or workaround? Appreciate.
... View more
Labels:
- Labels:
-
Apache Kudu
-
Apache YARN