@zeeshanmcs It seem you're having an issue with unavailable tablets in your Kudu table, which is preventing Spark from inserting data. The output from kudu cluster ksck clearly shows the problem: the leader replicas for all tablets in the impala::mrs.NumberofSubs table are on a tablet server that's unavailable. The key issue is that the tablet server with ID 24483fcd36ce45d78d80beb04b3b0cf4 is not running, and it happens to be the leader for all 7 tablets in your table. Here's a solution to resolve this issue: 1. First, check the status of all Kudu tablet servers sudo systemctl status kudu-tserver 2. Look specifically for the tablet server with ID 24483fcd36ce45d78d80beb04b3b0cf4 sudo -u kudu kudu tserver list tstewputil1 3. If the tablet server is down, start it. sudo systemctl start kudu-tserver 4. If the tablet server is running but not responding, restart it sudo systemctl restart kudu-tserver 5. After restarting the tablet server, wait a few minutes for it to rejoin the cluster and for leadership transitions to occur, then check the status again sudo -u kudu kudu cluster ksck tstewputil1 If the tablet server is permanently lost or damaged, you'll need to recover the tablets a. Check if you have enough replicas (you should have at least 3 for production) sudo -u kudu kudu table describe impala::mrs.NumberofSubs tstewputil1 b. If you have other healthy replicas, you can delete the failed server from the cluster and Kudu will automatically recover sudo -u kudu kudu tserver delete tstewputil1 <tablet_server_uuid> c. If this is the only replica and you don't have backups, you may need to: Create a new table with the same schema Load data from your source systems Or restore from a backup if available If, after restarting, you still have issues, the problem might be: Disk space issues on the tablet server Configuration problems Network connectivity problems between servers Check the Kudu tablet server logs for more details less /var/log/kudu/kudu-tserver.log Once the tablet server is back online and healthy, your Spark job should be able to insert data into the table successfully Happy hadooping
... View more
