Member since
01-16-2018
602
Posts
46
Kudos Received
100
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
13 | 04-15-2024 12:28 AM | |
20 | 04-15-2024 12:17 AM | |
47 | 04-15-2024 12:04 AM | |
19 | 04-14-2024 11:54 PM | |
26 | 04-14-2024 11:45 PM |
02-28-2023
08:43 PM
Hello @Menyawy Thanks for using Cloudera Community. The Error [1] indicates your Team is using CDH v5.x using HBase v1.x. In the concerned release, We have limited choices to ensure "hbase:namespace" Table Region is assigned correctly as shared below: (I) Easier Yet Not-Always-Successful Option: 1) Shutdown HBase Service
2) echo "rmr /hbase" | hbase zkcli (From HMaster Node)
3) Start HBase Service
4) sudo hbase hbase hbck -fixAssignments (II) Review whether any of the "hbase:namespace" Table Region files are reporting MissingBlockException. If Yes, We need to fix the MissingBlockException. (III) Possible for the RegionServer Logs (Wherein the "hbase:namespace" Table Region is being assigned) would offer additional insight into the reasoning for the RegionServer being Unable to assign the "hbase:namespace" Table Region successfully. Your Team may review the concerned RegionServer Logs as well. There may be Other Options, yet such Inconsistencies are better handled in HBase 2.x [1] & your Team should plan for using HBase v2.x soon. Regards, Smarak [1] https://github.com/apache/hbase-operator-tools/blob/master/hbase-hbck2/README.md#master-startup-cannot-progress-in-holding-pattern-until-region-onlined
... View more
02-28-2023
08:36 AM
2 Kudos
Hello @BrianChan Thanks for using Cloudera Community. I believe CDSW isn't included in Free Tier [0], Your Team should review CML (Cloudera Machine Learning), which is available in CDP Public Cloud & CDP Private Cloud with [1] offering details into the Difference between CDSW & CML. As noted in [1], CML expands the functionalities of CDSW further. CML is included in "CDP Public Cloud" Trail [2] & in "CDP Private Cloud Data Services" Trial [3]. Regards, Smarak [0] https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/installation/topics/cdpdc-setup-trial-cluster-using-wizard.html [1] https://docs.cloudera.com/machine-learning/1.3.4/product/topics/ml-cdsw-key-differences.html [2] https://www.cloudera.com/campaign/try-cdp-public-cloud.html [3] https://www.cloudera.com/downloads/cdp-private-cloud-trial.html
... View more
02-27-2023
10:59 PM
Hello @AmitBIDWH Thanks for using Cloudera Community. HBase offers Tools for Use-Cases as Bulk-Loading, Exporting, Importing, CopyTable etc. Additionally, few Task as Compactions are managed implicitly by Customer. We would suggest reviewing [1] for any HBase concerns. To your Q concerning Create > Admin > Load, BulkLoad [2] is one of the Best Tool for Large-Scale Loading. Additionally, Your Team may use HBase Shell or REST APIs [3] for further activities. Kindly review & let us know if you have any concerns. Regards, Smarak [1] https://hbase.apache.org/book.html [2] https://hbase.apache.org/book.html#arch.bulk.load [3] https://hbase.apache.org/book.html#_rest
... View more
02-27-2023
10:54 PM
1 Kudo
Hello @bgkim Thanks for using Cloudera Community. To your Q, the Composite Primary Key would require using both A & B in WHERE Clause as the Indexing is done collectively. As such, Your SELECT Query would ideally benefit upon creating a Local Index on A & C. You may review [1] as Read-Heavy Use-Case benefit via Global Index with Penalty incurred during Writes. Additionally, Phoenix offers Covered Index & Explain Plan helps confirming the Index Usage. Link [2] offers few examples as well. With all recommendations, Best Advise is always to review the Performance internally prior to implementing them in Production. Regards, Smarak [1] https://phoenix.apache.org/secondary_indexing.html [2] https://learn.microsoft.com/en-us/azure/hdinsight/hbase/apache-hbase-phoenix-performance
... View more
01-24-2023
11:30 PM
Hello @Ellengogo I am not able to find the exact Case reference as this Post is few months old. Typically, such issues are caused by Resource Constraint, wherein the Engine Pod (Created when a Workbench Session is started) gets terminated owing to Resource Constraint. If the issue is persistent, a Support Case would be ideal as we require the review of the Kubernetes Output pertaining to the Engine Pod along with Resource Profile & other related artefacts. Regards, Smarak
... View more
01-18-2023
12:10 AM
Hello @Girija Since we haven't heard back from you, We shall mark the Post as Solved. If you happen to have any further ask, Feel free to Update the Post. In Summary, Internally, I wasn't able to replicate the issue being faced by you as I was able to create the ConfigSet using "_default" ConfigSet as baseConfig. Customer can use the below solrctl command to create a ConfigSet with Solr KeyTab: solrctl config --create Test_Config _default -p configSetProp.immutable=false Assuming the above Command fails, Running the solrctl command with "--trace" after "solrctl" & before "config" would print the trace logging & assist in troubleshooting the issue faced by your team. Regards, Smarak
... View more
01-17-2023
11:03 PM
Hello @pankshiv1809 Since we haven't heard from your side concerning the Post, We are marking the Post as Solved. If you have any further ask, Feel free to update the Post & we shall get back to you accordingly. Regards, Smarak
... View more
01-17-2023
10:46 PM
Hello @mingtian Note that Debug Logging won't ensure the Balancer would perform Region Movement, rather the same would confirm if Balancer is running yet the same isn't moving any Region owing to CostFactor. Example: I ensured 1 RegionServer didn't had any Region by RegionMovement & triggered a Balancer, which showed [1] & trigger a Region Movement (Note "Found A Solution That Moves 1 Region"). After the 1st Balancer is Completed, I triggered a 2nd Balancer, which printed [2], wherein the DEBUG report "Skipping Load Balancing". I believe your Team would see [2] i.e. Balancer is Skipping any Load Balancing owing to Cost Factor. As such, Your Team can consider the fact that HBase is rejecting Region Movement owing to the fact that any new Region-Movement is "Costlier" than Current Region Placement. Tweaking [3] Cost Parameters including setting "hbase.master.loadbalance.bytable" to "true" should help trigger a Balancer for your Team. Regards, Smarak [1] 2023-01-18 06:38:33,290 INFO org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: Finished computing new moving plan. Computation took 95 ms to try 7200 different iterations. Found a solution that moves 1 regions; Going from a computed imbalance of 0.4961380973335763 to a new imbalance of 0.020487264673311183. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (not needed); MoveCostFunction : (multiplier=7.0, imbalance=0.3333333333333333, need balance); ServerLocalityCostFunction : (multiplier=25.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (not needed); RegionReplicaRackCostFunction : (not needed); ReadRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); WriteRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); [2] 2023-01-18 06:39:05,365 INFO org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: Cluster wide - skipping load balancing because weighted average imbalance=0.013858568086431631 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (not needed); MoveCostFunction : (multiplier=7.0, imbalance=0.0); ServerLocalityCostFunction : (multiplier=25.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (not needed); RegionReplicaRackCostFunction : (not needed); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.6685715976063684, need balance); WriteRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); [3] StochasticLoadBalancer (Apache HBase 3.0.0-alpha-4-SNAPSHOT API)
... View more
01-16-2023
09:09 PM
Hello @Ryan_2002 Thanks for engaging Cloudera Community. First of all, Thank You for the detailed description of the Problem. I believe your ask is Valid, yet reviewing the same over a Community Post isn't a suitable approach. Feasible for you to engage Cloudera Support to allow our Team to work with you, with the suitability of Screen-Sharing Session as well as Logs exchange, both of which aren't feasible in Community. That would greatly expedite the review of your ask. Regards, Smarak
... View more
01-16-2023
01:57 AM
Hello @mingtian Hope you are doing well. We wish to follow-up with you & check if the DEBUG Logging assisted in confirming the reasoning for Balancer Algorithm deciding against Region-Movement. If Yes, Kindly let us know if your Q in the Post has been answered or any further Q remains. Regards, Smarak
... View more