Member since
01-16-2018
540
Posts
33
Kudos Received
82
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
107 | 01-18-2023 12:10 AM | |
73 | 01-16-2023 01:54 AM | |
170 | 01-13-2023 01:59 AM | |
162 | 01-13-2023 01:35 AM | |
97 | 01-02-2023 10:03 PM |
02-01-2023
09:19 AM
I agree with Smarak, the error code typically means that there were not enough resources available to start the job. You could probably use Grafana dashboard (available in the Admin page) to look at the cluster resources and load around the time you had this issue. Is it happening consistently? For Jobs, I usually see this at the start or end of the month, when a lot of people schedule periodic jobs and they all trigger at the same time.
... View more
01-18-2023
12:10 AM
Hello @Girija Since we haven't heard back from you, We shall mark the Post as Solved. If you happen to have any further ask, Feel free to Update the Post. In Summary, Internally, I wasn't able to replicate the issue being faced by you as I was able to create the ConfigSet using "_default" ConfigSet as baseConfig. Customer can use the below solrctl command to create a ConfigSet with Solr KeyTab: solrctl config --create Test_Config _default -p configSetProp.immutable=false Assuming the above Command fails, Running the solrctl command with "--trace" after "solrctl" & before "config" would print the trace logging & assist in troubleshooting the issue faced by your team. Regards, Smarak
... View more
01-17-2023
11:03 PM
Hello @pankshiv1809 Since we haven't heard from your side concerning the Post, We are marking the Post as Solved. If you have any further ask, Feel free to update the Post & we shall get back to you accordingly. Regards, Smarak
... View more
01-17-2023
10:46 PM
Hello @mingtian Note that Debug Logging won't ensure the Balancer would perform Region Movement, rather the same would confirm if Balancer is running yet the same isn't moving any Region owing to CostFactor. Example: I ensured 1 RegionServer didn't had any Region by RegionMovement & triggered a Balancer, which showed [1] & trigger a Region Movement (Note "Found A Solution That Moves 1 Region"). After the 1st Balancer is Completed, I triggered a 2nd Balancer, which printed [2], wherein the DEBUG report "Skipping Load Balancing". I believe your Team would see [2] i.e. Balancer is Skipping any Load Balancing owing to Cost Factor. As such, Your Team can consider the fact that HBase is rejecting Region Movement owing to the fact that any new Region-Movement is "Costlier" than Current Region Placement. Tweaking [3] Cost Parameters including setting " hbase.master.loadbalance.bytable" to "true" should help trigger a Balancer for your Team. Regards, Smarak [1] 2023-01-18 06:38:33,290 INFO org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: Finished computing new moving plan. Computation took 95 ms to try 7200 different iterations. Found a solution that moves 1 regions; Going from a computed imbalance of 0.4961380973335763 to a new imbalance of 0.020487264673311183. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (not needed); MoveCostFunction : (multiplier=7.0, imbalance=0.3333333333333333, need balance); ServerLocalityCostFunction : (multiplier=25.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (not needed); RegionReplicaRackCostFunction : (not needed); ReadRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); WriteRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); [2] 2023-01-18 06:39:05,365 INFO org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: Cluster wide - skipping load balancing because weighted average imbalance=0.013858568086431631 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (not needed); MoveCostFunction : (multiplier=7.0, imbalance=0.0); ServerLocalityCostFunction : (multiplier=25.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (not needed); RegionReplicaRackCostFunction : (not needed); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.6685715976063684, need balance); WriteRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); [3] StochasticLoadBalancer (Apache HBase 3.0.0-alpha-4-SNAPSHOT API)
... View more
01-16-2023
09:09 PM
Hello @Ryan_2002 Thanks for engaging Cloudera Community. First of all, Thank You for the detailed description of the Problem. I believe your ask is Valid, yet reviewing the same over a Community Post isn't a suitable approach. Feasible for you to engage Cloudera Support to allow our Team to work with you, with the suitability of Screen-Sharing Session as well as Logs exchange, both of which aren't feasible in Community. That would greatly expedite the review of your ask. Regards, Smarak
... View more
01-16-2023
01:54 AM
Hello @panb We hope your Query has been addressed by us & shall mark the Post as Resolved. In Summary, Your Team needs to meet the requirement as stated in [1], which doesn't differentiate in Processor Type & I believe your Team is referring to Hygon Dhyana Processor. Note that we have shared the Hardware requirement is shared for CDP v7.1.8 as CDH isn't recommended now owing to End-Of-Life. As a Best Practise, I shall suggest engaging with Cloudera Account Team associated with Customer to perform any due diligence with respect to Supportability & Best Practices prior to onboarding Use-Cases into any new Platform, wherein Supportability is doubted by your Team. Regards, Smarak [1] Hardware Requirements | CDP Private Cloud (cloudera.com)
... View more
01-13-2023
01:43 AM
Hi Smarak, thanks for your answer. That helps me!
... View more
01-03-2023
08:46 AM
How u increased the timeout to 15 minutes ?
... View more
01-02-2023
10:03 PM
Hello @quangbilly79 Thanks for using Cloudera Community. Based on your Post, you may consider "Kafka Gateway" as the Client for Kafka, which are setup on the Hosts wherein the same is added as per Cloudera Manager "Assign Roles". A Client/Gateway is familiar with the Service (Kafka in this Case) & all Client/Service Configs are available for the Client/Gateway without any manual intervention. Any changes made to the Service or Client Configs is pushed to the Service/Client Configuration by Cloudera Manager. Imagine a Scenario wherein you wish to run "hdfs dfs -ls" on a HDFS FileSystem. Simply running the Command won't work unless the Host wherein the Command "hdfs dfs -ls" is being run knows the Setup (HDFS FileSystem, NameNode, Port, Protocol). Review [1] for an Example. Adding an HDFS Gateway ensures User doesn't need to manually configure a Client/Gateway with Cloudera Manager doing the needful. Similarly, Kafka Gateway operates. Else, Customer need to manually configure the Client/Gateway Setup. Hope the above answer your query concerning the Gateway Role. Regards, Smarak [1] https://www.ibm.com/docs/en/spectrum-scale-bda?topic=hdfs-clients-configuration
... View more
12-29-2022
02:47 AM
Hello @smdas , can you let me know or tag Hbase engineers who could provide more clarity on my doubts especially on the caching? It would be very helpful. Thanks
... View more
12-27-2022
11:27 PM
Hello @smdas Thanks for the response. https://issues.apache.org/jira/browse/HBASE-24289 https://docs.google.com/document/d/1fk_EWLNnxniwt3gDjUS_apQ3cPzn90AmvDT1wkirvKE/edit# These links mention date tiered compaction policy in hbase. Does it somehow help in configuring different policy for same column family? or did i misunderstood?
... View more
12-25-2022
05:50 AM
Hello @Serhii This is an Old Post, yet I am answering the same as there are few changes with CDP recent release & ensuring Community awareness. With CDP v7.1.6 [1] allows Accumulo to be installed via Cloudera Manager. The Installation is documented via [1] & requires a Separate Parcel to be installed before attempting to add Accumulo via Cloudera Manager. Having said that, Feel free to engage with Cloudera Account Team for Customer as the investment into Accumulo isn't as par with other similar counterpart to review any long term engagement with Accumulo for meeting Customer's Use-Case. Regards, Smarak [1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/opdb-accumulo-installation/topics/opdb-accumulo-install.html
... View more
12-21-2022
07:04 AM
1 Kudo
Hello @SagarCapG Confirmed that Phoenix v5.1.0 has the Fix for " !primarykeys" to show the Primary Key linked with a Phoenix Table. Upon checking our Product Documentation, CDP v7.1.6 introduces Phoenix v5.1.0 [1]. As such, I am surprised your Team has Phoenix v5.0.0 with CDP v7.1.7, wherein Official v7.1.7 Doc [2] says Phoenix v5.1.1.7.1.7.0-551 is used. Since the Issue is fixed in Phoenix v5.1.x & CDP v7.1.6 onwards ship Phoenix v5.1.x, Kindly engage Cloudera Support to allow Support to review your Cluster for identifying the reasoning for CDP v7.1.7 using Phoenix v5.0.0. Or, Upgrade to Phoenix v5.1.x (If Customer is managing Phoenix outside of CDP) to use "!primarykeys" functionality. Regards, Smarak [1] What's New in Apache Phoenix | CDP Private Cloud (cloudera.com) [2] Cloudera Runtime component versions | CDP Private Cloud
... View more
12-20-2022
11:35 PM
Hello @brajeshreddy Since the Issue isn't replicated with CML release internally & your team have engaged Cloudera Support for further assistance, We shall Close the Post now. For our fellow Community Users, the Steps to perform the Team's Name Modification is shared above. Assuming the same isn't working for Customers, Ensure you are connected as MLAdmin & Caching is ruled out as well. Regards, Smarak
... View more
12-20-2022
08:28 AM
Hello @sekhar1 Since we haven't heard back from your side concerning the Post, We shall mark the Post as Resolved with the Action Plan to review if Traffic from your Machine isn't allowed to or from the Security Group linked with the VPC wherein the CML Workspace instances are deployed. Check if the Kubernetes Pods associated with the CML Workspace Kubernetes Cluster are Up/Running. If Yes, Such Exception should be reviewed from a Network Standpoint only. You may reach out to your Customer's AWS/Platform Team to review the Traffic between your Machine & the VPC within which the CML Workspace is deployed. Assuming your team fixed the issue outside of any Customer's Network concerns, We would appreciate your feedback to ensure our fellow Community Users can benefit from your experience. Regards, Smarak
... View more
12-19-2022
12:34 AM
Hello @anks1 Thanks for using Cloudera Community. Kindly confirm the following details: CML Deployment (CDP Public Cloud Or Private Cloud) CML Version Used (Via CML Workspace Details In CML Page) Any timeframe of how many days before an App transition to Failed State from Running State Regards, Smarak
... View more
12-19-2022
12:15 AM
Hi @pacman A Merge happens for each Read Operation i.e. BlockCache & MemStore. As such, Incorrect Values aren't observed. Having said that, If you observe any such scenario of Read/Write Inconsistency, Kindly share a Use-Case & any replication attempt to allow us to review accordingly. Regards, Smarak
... View more
12-13-2022
12:48 AM
Hello @ashokaakula As per [1], the Master Password isn't recoverable in the event of the same being lost. As such, We shall request your Team to engage Cloudera Support, if your Team have lost the same without further delay. Regards, Smarak [1] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/navigator_encrypt_register.html#concept_opn_tnn_mr__section_g5n_tnn_mr
... View more
12-09-2022
07:14 AM
Hello @quangbilly79 Thanks for using Cloudera Community. The Link [1] applies to CDE Service, which is available on Private Cloud & Public Cloud. In CDP Public Cloud, Your Team need to connect to the CDP Management Console & access the CDE Service as documented in [1]. In CDP Private Cloud, You need to install Data Service [2] to access CDE Service, upon which your Team can Schedule Job. The Cloudera Manager of CDP Private Cloud is used to install Data Service [2] & CDE is available after successful installation on Data Service. Without Data Service, Oozie can be used by your Team as shared above by Steven. Kindly review & let us know if you have any queries. Regards, Smarak [1] Scheduling jobs in Cloudera Data Engineering [2] CDP Private Cloud Data Services: Documentation | CDP Private Cloud (cloudera.com)
... View more
12-02-2022
01:26 AM
Hello @GregDol Thanks for using Cloudera Community. This is an Old Post, yet I am sharing a review for other Community Users, in case they encounter the concerned issue. Reviewing the Post indicates the Issue is likely caused by missing " rt.jar" in the JDK used by the Solr Service. In other words, the concerned Class " java.time.zone.ZoneRulesProvider" is present in " /usr/java/jdk1.8.0_232-cloudera/jre/lib/rt.jar" for CDP Cluster (Path & Version may be different for Customer). The concerned JDK is used by the various Services running including Solr. In your scenario, the concerned Class appears to be missing. Assuming your team wish to confirm whether a JAR consists of the Class, you may use Command [ /usr/java/jdk1.8.0_232-cloudera/bin/jar -tvf /usr/java/jdk1.8.0_232-cloudera/jre/lib/rt.jar] to list the Classes, which should include "java.time.zone.ZoneRulesProvider" as well. If your Team is using any different JDK, Review the same & ensure the "rt.jar" is present for Class "java.time.zone.ZoneRulesProvider" to be loaded successfully. Regards, Smarak
... View more
12-01-2022
03:32 AM
Could you try to add 'home' at the end of provided URL ? It should be http://xxxxxx/airflow/home
... View more
12-01-2022
02:25 AM
Hello @QiDam As stated by JD above, the CDE Service relies on Ozone Service on Base Cluster. If Ozone Service wasn't in Healthy State, the CDE Service enabling would fail with similar tracing as shared by you. We would recommend the following checks: Ensure Ozone Service is Up & Running on Base Cluster. Create a new Environment & check CDE Service enabling on the new Environment. If the above CDE Service enabling on new Environment is Successful, Reattempt the CDE Service enabling on the existing Environment. If the above Suggestion doesn't help, We would suggest engaging Support as any further troubleshooting would require sharing the Logs over the Public Community forum, which may have Customer's details. We shall mark the Post as Resolved now. If you have any concerns, Feel free to Update the Post likewise. Regards, Smarak
... View more
11-21-2022
02:24 AM
1 Kudo
Hello @sfdragonstorm & @pacman In the Region Name "img,0006943d-20150504220458043384375D00000002-00093,1527295748538.7b45a9f6f5584fc50b3152d41a5323a2.", the Table Name is "img", StartKey is "0006943d-20150504220458043384375D00000002-00093", Timestamp is "1527295748538" & "7b45a9f6f5584fc50b3152d41a5323a2" is the Region ID. Under HBase Data Directory, each Table Directory would have Region-Level Directories as identified by Region ID ("7b45a9f6f5584fc50b3152d41a5323a2" in Example). The Region ID is an MD5 encoded string for the Region Name & generated by HBase itself. Refer [1], if your Team wish to review the same. Regards, Smarak [1] https://hbase.apache.org/apidocs/src-html/org/apache/hadoop/hbase/client/RegionInfo.html#line.164
... View more
11-15-2022
11:20 PM
I finally engaged Cloudera support. After checking the security value with the following command : /var/lib/ambari-server/resources/scripts/configs.py -l <ambari host fqdn> -t <port> -u admin -p <password> -n <cluster_name> -a get -c cluster-env | grep -i security_enabled, I was asked to disable the security_enabled : /var/lib/ambari-server/resources/scripts/configs.py -l <ambari host fqdn> -t <port> -u admin -p <password> -n <cluster_name> -a set -c cluster-env -k security_enabled -v false Then i was able to start ranger again Still have problem with infa_solr. I will engage support for that
... View more
11-15-2022
09:13 PM
Hello @balance002 Kindly let us know if the above Post dated 2022-11-08 assisted your team to identify the affected Hfile(s) & take necessary remediation to ensure Table's access is restored for Users. Regards, Smarak
... View more
11-15-2022
09:11 PM
Hello @hanumanth Hope you are doing well. Kindly confirm if your ask has been addressed by our Post dated 2022-11-09. If Yes, Please mark the Post as Solved. If No, Feel free to share any further queries/concerns & we shall respond accordingly. Regards, Smarak
... View more
11-09-2022
10:41 PM
Hello @TheFixer We hope your queries around the RegionServer log traces has been answered by us. As such, We shall mark the Post as Solved. If you have any further ask, Feel free to update the Post & we shall get back to you accordingly. Regards, Smarak
... View more
11-09-2022
10:39 PM
Hello @Venkatd Since this is an Old Post & we have shared a few Action Plan with missing "atlas.application.properties" from Atlas Configuration Directory in HBase Configuration Directory "/etc/hbase/conf" being the Most-Likely reason for the concerned failure along with Steps to mitigate the HMaster/RegionServer failure (By setting "hbase.coprocessor.abortonerror" as "false") to avoid any Co-Processor failure cause Master/RegionServer startup failure, We shall mark the Post as Solved now. Thank You for using Cloudera Community. Regards, Smarak
... View more
11-09-2022
08:51 AM
1 Kudo
Hello @TouguiOmar Update acknowledged. You may wish to use Step (VI) from above Action Plan & share the Outcome. Regards, Smarak
... View more
11-08-2022
11:12 PM
Hello @sekhar1 Thanks for using Cloudera Community. The Error received by you is caused by a missing entitlement. For Customer without the concerned entitlement, the Error is received. We are improving the Product Experience to ensure the Error isn't displayed for Customer missing the entitlement. In your Case Or any one receiving the error, Kindly engage Cloudera Support to enable an entitlement to avoid getting the concerned error. Since Entitlement enabling requires few Customer's specific details, the Support engagement path is recommended. Regards, Smarak
... View more