Skip to main content
itsmezeeshan
itsmezeeshan
Silver ⭐️⭐️
December 13, 2023
Question

Kudu nodes are connected but no data is showing

using cdp 7.14 

kudu is working and nodes are connected but giving no data 
ksck error :  

Wed Dec 13 15:51:29 +03 2023
JAVA_HOME=/usr/java/jdk1.8.0_232-cloudera
Using /var/run/cloudera-scm-agent/process/1546472905-kudu-KUDU_MASTER-kudu_master_ksck_cmd as conf dir
Using scripts/kudu.sh as process script
CONF_DIR=/var/run/cloudera-scm-agent/process/1546472905-kudu-KUDU_MASTER-kudu_master_ksck_cmd
CMF_CONF_DIR=
Wed Dec 13 15:51:29 +03 2023: KUDU_HOME: /apps/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/lib/kudu
Wed Dec 13 15:51:29 +03 2023: CONF_DIR: /var/run/cloudera-scm-agent/process/1546472905-kudu-KUDU_MASTER-kudu_master_ksck_cmd
Wed Dec 13 15:51:29 +03 2023: CMD: ksck
Wed Dec 13 15:51:29 +03 2023: Found master(s) on m507,m508,m01
Master Summary
UUID | Address | Status
----------------------------------+---------+---------
0a0209b49d6e4f709221336403bae183 | m507 | HEALTHY
127407e626be4f4e9290389b1bfc50f3 | m508 | HEALTHY
580630447da74d788de916544b487399 | m01 | HEALTHY

Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system | all 3 server(s) checked

Tablet Server States
Server | State
----------------------------------+------------------
12716b1c8e7e4981ad3f7845b8281461 | MAINTENANCE_MODE

Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------+---------+----------+----------------+-----------------
1c6b5d57809d4383b75a88796c9e7842 | m01:7050 | HEALTHY | /default | 232 | 0
95b372778c944001847538a57a483e4e | m04:7050 | HEALTHY | /default | 0 | 0
b006242a79d04dcba28ab39c197460b5 | m02:7050 | HEALTHY | /default | 0 | 0
dc6582dfb64d4993b0515632117ddf7c | m03:7050 | HEALTHY | /default | 0 | 0

Tablet Server Location Summary
Location | Count
----------+---------
/default | 4

Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system | all 4 server(s) checked

Version Summary
Version | Servers
--------------------+-------------------------
1.15.0.7.1.7.0-551 | all 7 server(s) checked

Tablet Summary
Tablet 0ab274df35a24a028896f0a69342257a of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 28f35408192d4e9babb5eeb8cfe216d6 of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 1e82c80980004e31b0dc43067275094d of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 4ffc4da9bd19493fbf662c58172ed08b of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 20ab7281edc94477a3b1dbd8b21eb81a of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 4c7ef4d918b14ec9ad44b0a99f9c4e66 of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet c9c1355b006047c1900d0b6ccd73421a of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 693c278f77214ae4bdd7bcea922d7c59 of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 724c8a4385b144e0b3a3ba23f4c24262 of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 7fc30745a129488cb108a236a458d0d6 of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 0ad16348079244c98bc313896b0a1f9c of table 'impala::mobilysemantic_UAT.TransactionSummaryValueAgg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet e4d14cee21874540808e1cffb46a4e1b of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet fcb079859b57416b9b2723db848a9df5 of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 05fe87253a834dc99fc0c63bda6d8c7c of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 9697ccc34f224cf99577003fd0f63852 of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 236118f99e2249d6a32f587013235ae0 of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet ca118cc9bfa24410bab344c2ce831968 of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 5d0fb5e15e8146e78ae38e3b89a43846 of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet f89284d2163c49239cb158074d2097ce of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 758edce1ad434d7a836188cf2d56f79e of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 631065d39aa94fad9203fc826cf26466 of table 'impala::mobilysemantic_UAT.useraccountbalance_agg' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 78dd298ebfce4c1facf5df1b0ebb731f of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 736ebc2c4bd34c69b8504a4cae8195ad of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 0c9cbca6cd1446faa747924d121a8bd2 of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 0802795dc577428484f547770189d708 of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet fcc821f8f35c4d569056bed347359ab0 of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 64ef3de59e674ebca05cdd2e4acc805d of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet ab43e09f6ff243ccbcc9a2703188cee2 of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 19d4af690b2d4ea8bde679288157d1f0 of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 7253c4549cd34752ad9898190720f2db of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet a35f7804bea14cb3a858fbcc11f429a9 of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet d83b0810aa53441fbb092f3d45f06089 of table 'impala::mobilysemantic_UAT.WalletDataCopy' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 0075c675ad7d493288bb1af647f489cf of table 'impala::mobilysemantic_UAT.BlockedAccountsAndFraudDevices' is unavailable: 1 replica(s) not RUNNING
12716b1c8e7e4981ad3f7845b8281461: TS unavailable [LEADER]
All reported replicas are:
A = 12716b1c8e7e4981ad3f7845b8281461
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 420aaf6814c4417289d84a5cc85c4410 of table 'impala::mobilysemantic_UAT.BlockedAccountsAndFraudDevices' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 796b756750ed42aba865bfec464a6f4b of table 'impala::mobilysemantic_UAT.BlockedAccountsAndFraudDevices' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet 792d68dab55f494db8fff2c7300f909a of table 'impala::mobilysemantic_UAT.BlockedAccountsAndFraudDevices' is unavailable: 1 replica(s) not RUNNING
e7ab31426d1c4ea3b90569bd6fdb3ef7: TS unavailable [LEADER]
All reported replicas are:
A = e7ab31426d1c4ea3b90569bd6fdb3ef7
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* | | | Yes
A | [config not available] | | |

Tablet b89aa3fe595345748a2df75d9defb957 of table 'impala::mobilysemantic_UAT.BlockedAccountsAndFraudDevices' is unavailable: 1 replica(s) not RUNNING
822f79d744a243b3b1258d73c5e9f3d1: TS unavailable [LEADER]
All reported replicas are:
A = 822f79d744a243b3b1258d73c5e9f3d1

1 reply

Rajat_710
Gold ⭐️⭐️
Rajat_710
Gold ⭐️⭐️
April 22, 2024

Hi, @itsmezeeshan the graph data will be populated by the Service Monitor, you need to check the SMON logs for that and also If the number of events is more in your cluster you might need to increase SMON java and non-java heap memory for the processing for all the events and turn them into charts data. 

Translate
Terms & ConditionsAccessibility statement