Member since
07-30-2020
219
Posts
45
Kudos Received
60
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
435 | 11-20-2024 11:11 PM | |
488 | 09-26-2024 05:30 AM | |
1084 | 10-26-2023 08:08 AM | |
1852 | 09-13-2023 06:56 AM | |
2129 | 08-25-2023 06:04 AM |
11-07-2022
01:43 AM
Hi @lysConsulting The No route to host indicates that there is a network issue between this client and the HDFS cluster. I would suggest to check if the basic networking is working fine between these hosts.
... View more
11-01-2022
06:18 AM
HI @Felix-Han Based on the Error, it seems there are too many request coming to that specific Region server thereby consuming the handlers it has. So may want to check if multiple clients are hitting the same Region server ( hotspot) or try to raise the handler count. https://my.cloudera.com/knowledge/CallQueueTooBigException--Call-queue-is-full-on-000060020-too?id=73901
... View more
10-31-2022
05:28 AM
1 Kudo
Nope. You will need to increase the GC threads. CMSInitiatingOccupancyFraction decides when to perform a Full GC and default value afaik in HDP is at 92%. Further, the reason for 2 sec pauses are very hard to identify.
... View more
10-29-2022
12:04 AM
You can also try to split the table at the time of creation or split it now so that not all requests go to the same Region server which thereby creates a hotspot and might be a bottleneck.
... View more
10-28-2022
03:13 AM
1 Kudo
Hi @mazz , Check if the below article helps you in achieving this. You will need to modify the duration. https://community.cloudera.com/t5/Support-Questions/How-to-do-a-cleanup-of-hdfs-files-older-than-a-certain-date/m-p/182146
... View more
10-28-2022
03:07 AM
Those would go into hadoop-env inside HDFS. https://docs.cloudera.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-operations/content/tuning_garbage_collection.html
... View more
10-28-2022
01:33 AM
2 Kudos
Hi @mike_bronson7 , The thumb rule is to have 1 Gb of heap allocated to 1 Million block. Now, as you have already doubled the heap, I think you can check on improving the garbage collector tunings. https://community.cloudera.com/t5/Community-Articles/NameNode-Garbage-Collection-Configuration-Best-Practices-and/ta-p/245276 You can try tunings the GC threads to see if that helps. Been said that, these pauses are not very huge to cause any real trouble and we don't see it repeating very frequently ( within mins) and can be expected on a busy Cluster. You can also check if the threshold to report a JVM pause alert can be adjusted to say 5-10 secs which needs some intervention.
... View more
10-28-2022
01:23 AM
Hi @TheFixer , During read, hbase has to fetch the blocks from other slave nodes if the locality of the regions are not good and this can contribute to latency as the block in not local to the client (Hbase ) and has to go over the network to fetch those provided the disks are fast enough. So you may want to run a major compaction on that table to see if that improves the read performance. Further, when the reads are performed the 1st time on a table that is written recently, the blocks are not cached in memory. So subsequent reads should give good performance comparatively as the blocks will be cache in the BlockCache.
... View more
10-21-2022
04:14 AM
Hi @hanumanth , The network bandwidth needs to be limited at the OS side using the tools such as traffic control (tc) which can be set on the NIC which carries the ip address for the CDH roles. For info on this can be found in the Red Hat docs : https://access.redhat.com/solutions/69133 https://access.redhat.com/solutions/1324033
... View more
10-18-2022
09:41 AM
Hi @fengsh Yes, The remove script indeed has few bugs using curl which was picked up from AMBARI-18435 but this curl call is not officially supported and as such we don't have an official doc on this. The other option to remove the older stack with "yum remove <version>" command, specifying the exact version you want to remove should get the job done. You can also verify the list of packages to be removed using (please be thorough!): # yum list installed | grep -P 'version' Additionally, please verify the symlinks in the "/usr/hdp/current/" folder are pointing to the correct location. -- Was your question answered? Please take some time to click on “Accept as Solution” below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more