Hi @yagoaparecidoti  The log4j CVE-2021-44228 is fixed in 7.1.7 SP1 https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/runtime-release-notes/topics/rt-pvc-cve-sp1.html ... View more

Hi @kkanchu  Can you please accept the answer if the above steps helps to change the log level of the HBase daemon on the fly without any restart. ... View more

We can use the HBase master UI for enabling and disabling the TRACE/DEBUG/INFO without restarting any service.   ➤ To enable the TRACE/DEBUG logging please follow the below steps from the Hbase master UI for the master server. Step 1: Hbase Master UI -> click on log level-> add the property(org.apache.hadoop.ipc) on the second log box-> enter the DBUG in Level box-> click on Set log level radio button.   ➤ To enable the TRACE/DEBUG logging please follow the below steps from the Hbase master UI for the specific region server. Step 1: Hbase Master UI -> select specific region server-> click on log level-> add the property(org.apache.hadoop.ipc) on the second log box-> enter the DBUG in Level box-> click on Set log level radio button.   ➤ Once the completion of the above steps, just verifies whether the changes are effected or not with the below steps. Step 2: Hbase Master UI -> select specific region server/Master server-> click on log level-> add the property(org.apache.hadoop.ipc) on the fist log box-> click on the Get log level radio button, and verify the log level.   Or    ➤ hadoop daemonlog -setlevel <hbase regionserver host>:<port> org.apache.hadoop.hbase TRACE/DEBUG/INFO ... View more

Hi, @kras Thank you for writing back with your observation. Can you please check the below details as well? 1) When the Region Server JVM reports High CPU, Open "top" Command for the Region Server PID, 2) Use "Shift H" to open the Thread View of the PID. This would show the Threads within the Region Server JVM with CPU Usage, 3) Monitor the Thread View & Identify the Thread hitting the Max CPU Usage, 4) Take Thread Dump | JStack of Region Server PID & Compare the Thread with the "top" Thread View consuming the Highest CPU. 5) Check the CUP usage of the other services that are hosted on the Region Server host.   The above Process would allow you to identify the Thread contributing towards the CPU Usage. Compare the same with other Region Server & your Team can make a Conclusive Call to identify the reasoning for CPU Utilization. Howsoever Logs are reviewed, Narrowing the Focus of JVM review would assist in identifying the Cause. Review shared Link for additional reference.   Ref:  https://www.infoworld.com/article/3336222/java-challengers-6-thread-behavior-in-the-jvm.html  https://blogs.manageengine.com/application-performance-2/appmanager/2011/02/09/identify-java-code-co...  https://blog.jamesdbloom.com/JVMInternals.html   Thanks & Regards, Prathap Kumar. ... View more

There are many reasons this could happen including OS/Kernel bugs (update your system), swap, transparent huge pages, pauses by a hypervisor for the High latency issues.   1) As we are seeing the "responseTooSlow" on the region servers, please check the data node logs for the underlying issue from the data node logs. 2) In the data node logs please check we have below ERROR/WARN in the data node logs are not. Slow BlockReceiver write data to disk cost - This indicates that there was a delay in writing the block to the OS cache or disk. Slow BlockReceiver write packet to mirror took - This indicates that there was a delay in writing the block across the network Slow flushOrSync took/Slow manageWriterOsCache took - This indicates that there was a delay in writing the block to the OS cache or disk 3) If we have the above ERROR/WARN we need to check the infra team and OS vendor team to fix the underlying hardware issues to overcome issue.   ... View more