Hello Adar, Formatting is not the actual solution. In place of formatting i can delete. so similar way there are other problems i.e.TABLET_DATA_COPYING  and  Tablet Copy: Downloading WAL segment with seq https://github.com/apache/kudu/blob/master/src/kudu/tserver/tablet_copy_client.cc I just see this as peer as Learner and the tablet data was not copying and stuck in  TABLET_DATA_COPYING. Also see the Error . IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: Could not remove renamed recovery dir   I use the temp option to change the leader but this is not solution to correct it. As still the replica is in learner state not follower. So need help for similar kind of issues, how can resolve it by looking into code. It also through the error --> Tablet copy aborted. I saw inside code and found that error in tablet copy client file.  Kudu Developer will be definitely aware how to resolve these kind of issues as they are fully aware about code flow where it stuck.   So need that kind of help for the issue.         ... View more

Thanks Adar, Need some clarification because looking into Design doc doesn't help much.  i.e. For kudu file issue troubelshooting i try to look into cfile_reader.cc   Status CFileReader::Init () { Substitute("failed to init CFileReader for block $0",block_id().ToString()));   We got issue in cluster org.apache.kudu.client.NonRecoverableException: failed to init CFileReader for block : bad CFile header magic: I\x19[\xd6(!xE\xe1\xe0n\x8f   Now how i can work as temp solution . If i can get clarity what this code does and how i can debug this and patch for alternate solution, that gone help me. So if i will be aware about this code then i can troubleshoot lot of problem.   That is the objective of understanding the code. Regards Kamal     ... View more

I am getting service monitor alerts from long time but these YARN specific alerts come to me form last week. some of the information which I am getting....   org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary: appId=application_1555641507362_184814,name=INSERT OVERWRITE TABL...Test(Stage-1),user=hive,queue=root.van,state=FINISHED,trackingUrl=myhost:8088/proxy/application_1555641507362_184814/,appMasterHost=XYZ,startTime=1557309811366,finishTime=1557310013621,finalStatus=SUCCEEDED,memorySeconds=13123252,vcoreSeconds=2985,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=<memory:0\, vCores:0> 2019-05-08 05:06:59,921 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager: Max number of completed apps kept in state store met: maxCompletedAppsInStateStore = 10000, removing app application_1555641507362_174834 from state store. 2019-05-08 05:06:59,921 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager: Application should be expired, max number of completed apps kept in memory met: maxCompletedAppsInMemory = 10000, removing app application_1555641507362_174834 from memory: 2019-05-08 05:06:59,921 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Removing info for app: application_1555641507362_174834 2019-05-08 05:07:00,047 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_e64_1555641507362_184829_01_000007 Container Transitioned from RUNNING to COMPLETED 2019-05-08 05:07:00,047 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: Completed container: container_e64_1555641507362_184829_01_000007 in state: COMPLETED event:FINISHED       ... View more

Hello Sid, After going through the active Resource managers log I didn't find anything.Just the Info logs appeared , no warning or error logs there. what more need to check now.   ... View more

i.e. If i receive the alert with Activity Monitor. Pause Duration Bad Average time spent paused was 2 minute(s), 54 second(s) (290.37%) per minute over the previous 5 minute(s). Critical threshold: 60.00%.   There are various charts like disk latency,Disk throughput,networkthroughput. Garbage collection time.  How I can understand due to which this problem occurs in system. Will any of the specific charts help me there.     ... View more

i.e. Hive Metastore Canary Duration. ... View more

Hello  manuroman, How i can map this information to Charts. i..e.  Hive canary chart. How i can map if the charts are associated with an issue. Any detailed information related to charts is required by me. How to corelate an issue with charts. ... View more