Member since
07-30-2020
219
Posts
46
Kudos Received
60
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4887 | 11-20-2024 11:11 PM | |
| 2928 | 09-26-2024 05:30 AM | |
| 2459 | 10-26-2023 08:08 AM | |
| 4196 | 09-13-2023 06:56 AM | |
| 4509 | 08-25-2023 06:04 AM |
12-06-2022
12:49 AM
Hi @hanumanth , As you can see, the spark job is trying to reach the zookeeper on the localhost. 22/12/05 22:30:12 WARN zookeeper.ZKUtil: hconnection-0x5e29988e0x0, quorum=localhost:2181 We expect a zookeeper quorum of 3 or more ZK under quorum=. So this indicates that the node on which the spark job is running doesn't have a hbase-site.xml to direct the job to use the hbase.zookeeper.quorum. So make sure you have a Hbase Gateway role deployed on the node from where you are running the spark job and also try running the job if spark-submit using for eg "--files /etc/spark/conf/yarn-conf/hbase-site.xml"
... View more
11-23-2022
10:50 PM
1 Kudo
HI @bgkim, Hbase won't work well with data folders of hbase set to EC policy. EC did not implement hflush which is required to not lose data. Additionally, the Data locality is also killed with EC which makes it slower. https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/cdppvc-data-migration-opdb/topics/cdppvc-data-migration-hbase-unsupported-interfaces-features.html -- Was your question answered? Please take some time to click on “Accept as Solution” below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
11-23-2022
08:25 AM
1 Kudo
Hi @bhushan04ec041 , Can you try the below and check : 1) Stop the Oozie service 2) Select the "Delete" option from drop down menu. Under "Remove service Dependencies", select "Configure Service Dependency" and uncheck "oozie". 3) Save Changes 4) Delete Oozie
... View more
11-17-2022
01:19 AM
@mike_bronson7 -- Was your question answered? Please take some time to click on “Accept as Solution” below this post.
... View more
11-07-2022
01:43 AM
Hi @lysConsulting The No route to host indicates that there is a network issue between this client and the HDFS cluster. I would suggest to check if the basic networking is working fine between these hosts.
... View more
11-01-2022
06:18 AM
HI @Felix-Han Based on the Error, it seems there are too many request coming to that specific Region server thereby consuming the handlers it has. So may want to check if multiple clients are hitting the same Region server ( hotspot) or try to raise the handler count. https://my.cloudera.com/knowledge/CallQueueTooBigException--Call-queue-is-full-on-000060020-too?id=73901
... View more
10-31-2022
05:28 AM
1 Kudo
Nope. You will need to increase the GC threads. CMSInitiatingOccupancyFraction decides when to perform a Full GC and default value afaik in HDP is at 92%. Further, the reason for 2 sec pauses are very hard to identify.
... View more
10-29-2022
12:04 AM
You can also try to split the table at the time of creation or split it now so that not all requests go to the same Region server which thereby creates a hotspot and might be a bottleneck.
... View more
10-28-2022
03:13 AM
1 Kudo
Hi @mazz , Check if the below article helps you in achieving this. You will need to modify the duration. https://community.cloudera.com/t5/Support-Questions/How-to-do-a-cleanup-of-hdfs-files-older-than-a-certain-date/m-p/182146
... View more
10-28-2022
03:07 AM
Those would go into hadoop-env inside HDFS. https://docs.cloudera.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-operations/content/tuning_garbage_collection.html
... View more