HI Team, if you are storing the back in HDFS like above -rootPath file:/// [***DIRECTORY TO USE FOR BACKUP***]  then use the below command: hdfs dfs -du -s -h file:/// [***DIRECTORY TO USE FOR BACKUP***]  if its different storage like ozone(ofs/o3) then also hdfs command will work if its S3 then use the aws command ... View more

Hi Team, It does looks like too much data is writting to TS which causing it to reach 100%, did you check from TOP command how much kudu process is taking To fix it: 1. Please us the API in batch mode dont run all the jobs at one go as kudu works better in batch process 2. if possible reduce the load via increasing TS and data disk 3. Also check if any TP is running like antivirus which is not recommend and can cause a huge spike in CPU ... View more

1. Does it reduce the memory consumption once restart of this server 2. Leader master will have more consumption but in your case its look like a bug to me What is the CDH/CDP version you are using ... View more

Its Because of this Disk change where UUID of this TS was different but after change of WAL it created new one hence you seeing the wrong_server_UUID , simple restart of this TS will fix the issue in most of cases if not then please rebuild this TS from scratch while deleting the data and wal dir of this one it will solve the issue Note: Only proceed with rebuild if you have RF=3 in the cluster otherwise it will be a dataloss scenario ... View more

Hi Team, Leader master is the one who takes all front hit coming from client or connection so its obvious the consumption of Leader will be high ... View more

Hi Team as per the issue Master A is the fault one because once its in line there is no election happening please follow the below step to sync it you need to cleansafe the data from this master and replicate it back from leader master A)stop the problematic Master B)sudo -u kudu kudu local_replica delete --fs_wal_dir=/wal/kudu/wal --fs_data_dirs=/wal/kudu/data 00000000000000000000000000000000 -clean_unsafe C)sudo -u kudu kudu local_replica copy_from_remote --fs_wal_dir=/wal/kudu/wal --fs_data_dirs=/wal/kudu/data 00000000000000000000000000000000 <active_leader_fqdn>:7051 D)stop remaining two masters E)then start all the three masters. ... View more

HI Team, if you have 1 TS which is using for read and write then its obvisous to see the slowness from disk I/O as read/write both are taking the BW, You should be checking the kuud TS logs if you seeing any backpressure, soft memory exceed, any timeout, ancestral memory these all parameter will tell you if there is overload to TS and you can increase the parameter as needed, Please remmember running 1 TS with 848 tablets is fine but you should atleast have 3 TS so that one TS will not do all the work ... View more

HI Team, you can check the mount using below: showmount -e <NFS servername> Although as per the error it OS issue you should be checking with your internal team ... View more

HI Team with CDP cluster there is no such test has been done with 3rd party storage also on CDP only a couple of S3  storage options are currently considered (IBM and Dell) ... View more

Hi Team, it does looks like a syntax error "Empty key" could you please confirm the API you are passing has the key can you share it here  ... View more