Hello @Malvin    We are marking the Post as Solved as Region Server Grouping is the Only Way to Segregate the Region Assignment at Region Server Level. If your Team encounter any issues, Feel free to Update the Post & we shall get back to your Team accordingly.   Thank You again for using Cloudera Community.   Regards, Smarak ... View more

Hello @rahuledavalath    We are marking the Post as Solved. If your Team have any further concerns, Please update the Post, if your Team face any issues with either of the Recommendation shared [Migrate the System Tables Or Create a Phoenix Table afresh on Top of HBase Table].   Thank You for using Cloudera Community.    Regards, Smarak ... View more

Hello @Ma_FF    Thanks for using Cloudera Community. Based on the Post, 1 Region Server is using High CPU. As requested by @PrathapKumar, Review the same. Additionally, Your Team can perform the below:   (I) When the Region Server JVM reports High CPU, Open "top" Command for the Region Server PID, (II) Use "Shift H" to open the Thread View of the PID. This would show the Threads within the Region Server JVM with CPU Usage, (III) Monitor the Thread View & Identify the Thread hitting the Max CPU Usage, (IV) Take Thread Dump | JStack of Region Server PID & Compare the Thread with the "top" Thread View consuming the Highest CPU.    The above Process would allow you to identify the Thread contributing towards the CPU Usage. Compare the same with other Region Server & your Team can make a Conclusive Call to identify the reasoning for CPU Utilization. Howsoever Logs are reviewed, Narrowing the Focus of JVM review would assist in identifying the Cause. Review shared Link for additional reference.   Kindly review & share your Observation in the Post.    Regards, Smarak   [1] https://www.infoworld.com/article/3336222/java-challengers-6-thread-behavior-in-the-jvm.html [2] https://blogs.manageengine.com/application-performance-2/appmanager/2011/02/09/identify-java-code-consuming-high-cpu-in-linux-linking-jvm-thread-and-linux-pid.html [3] https://blog.jamesdbloom.com/JVMInternals.html   ... View more

Hello @amit_hadoop    Thanks for using Cloudera Community. Based on the Post, You installed HDP Sandbox 3.0 & observed Timeline Service v2 Reader reporting [1].    If you observe the Port mentioned, It says "17020" & HBase RegionServer runs with "16020" Port by default. YARN Timeline Service uses an Embedded HBase Service, which isn't equivalent to HBase Service as we may see in the Ambari UI. Kindly let us know the Outcome of the below Steps:   Whether Restarting YARN Timeline Service (Which restarts the Embedded HBase Service) helps, Within the YARN Logs, We should have an Embedded HBase Logs for HMaster & RegionServer. Reviewing the Logs may offer additional clue into the reasoning for Embedded HBase concerns, Review [2] for similar concerns wherein the ZNode was Cleared & Service Restarted, Review [3] for Documentation on Deleting the ZNode & Data. Restarting the YARN Timeline Service should initialise the HBase Setup afresh.    Regards, Smarak   [1] Error Trace: 2021-09-29 05:47:58,587 INFO [main] client.RpcRetryingCallerImpl: Call exception, tries=6, retries=36, started=5385 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server sandbox-hdp.hortonworks.com,17020,1632894462010 is not running yet details=row 'prod.timelineservice.entity' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=sandbox-hdp.hortonworks.com,17020,1628801419880, seqNum=-1 [2] https://community.cloudera.com/t5/Support-Questions/Yarn-Timeline-Service-V2-not-starting/td-p/281368 [3] https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/data-operating-system/content/remove_ats_hbase_before_switching_between_clusters.html     ... View more