About smdas

smdas · ‎01-13-2023

Hello @Ryan_2002 Thanks for using Cloudera Community. To your Q, the Driver Cap is the Engine/Resource Profile & the Executor's Resource Usage is defined by the SparkSession or "spark-defaults.conf" file within the Project wherein the Workbench Session is being created. Your Team can review the Pods in the User's Namespace & see the same i.e. upon a Workbench Session Creation, an Engine Pod is started with "Limits" set toEngine/Resource Profile Settings. After SparkSession is initialised, additional Pods are generated within the User's Namespace based on the Execution's Configs passed via SparkSession or "spark-defaults.conf" file. You may configure the Executor's Configs as per your usage yet the same depends on the CML Workspace AutoScale Range & InstanceType. Say, an InstanceType supporting 8 vCPU & Executors requesting 8 vCPU won't work. Similarly, AutoScale Max of 5 yet requesting Executors collectively utilising the Resource Limit of 5 Nodes. Hope the above helps answer your Post's queries. If Yes, Kindly mark the Post as Solved. If No, Feel free to share your concerns & we shall address accordingly. Regards, Smarak

smdas · ‎01-02-2023

Hello @quangbilly79 Thanks for using Cloudera Community. Based on your Post, you may consider "Kafka Gateway" as the Client for Kafka, which are setup on the Hosts wherein the same is added as per Cloudera Manager "Assign Roles". A Client/Gateway is familiar with the Service (Kafka in this Case) & all Client/Service Configs are available for the Client/Gateway without any manual intervention. Any changes made to the Service or Client Configs is pushed to the Service/Client Configuration by Cloudera Manager. Imagine a Scenario wherein you wish to run "hdfs dfs -ls" on a HDFS FileSystem. Simply running the Command won't work unless the Host wherein the Command "hdfs dfs -ls" is being run knows the Setup (HDFS FileSystem, NameNode, Port, Protocol). Review [1] for an Example. Adding an HDFS Gateway ensures User doesn't need to manually configure a Client/Gateway with Cloudera Manager doing the needful. Similarly, Kafka Gateway operates. Else, Customer need to manually configure the Client/Gateway Setup. Hope the above answer your query concerning the Gateway Role. Regards, Smarak [1] https://www.ibm.com/docs/en/spectrum-scale-bda?topic=hdfs-clients-configuration

smdas · ‎12-27-2022

Hello @sachin_saju Thanks for using Cloudera Community. You have 2 ask in the Post: 1. How to configure different Storage Policies with Cold & Hot Data, 2. Applying different Compression Algorithm in 1 Column Family. For Q2, I believe the same isn't feasible i.e. Compression Algorithm can be set at CF level. Review [1] for the Compression Algorithm recommendation around Hot & Cold type data. For Q1, I assume you are referring to HDFS Storage Policy. If Yes, the same is configured uniformly i.e. I am not sure if we can apply different HDFS Storage Policy for different data within the same CF. In HBase, We generally recommend SSD [2] for WAL, else the HBase Data relies on HDFS Storage Policy used. Alternatively, Use BackUp-Restore [3] for having a "Cold" Version of Data, which can be restored as per requirement. Regards, Smarak [1] https://hbase.apache.org/book.html#data.block.encoding.types [2] https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/configuring-hbase/topics/hbase-configure-storage-policy-wal.html [3] https://hbase.apache.org/book.html#br.overview

smdas · ‎12-27-2022

Hello @sachin_saju Thanks for using Cloudera Community. Your queries concerning the Read Path is discussed between a fellow Community User & myself in [1]. Kindly review the same & let us know if the same answer the queries around Read Path. In Summary, Read Path relies on a Merge of BlockCache & MemStore prior to returning the Output to the End-User, thereby avoiding any Inconsistent Read. Refer [2] for few Diagram around the same to help explain the Read Merge Path. Concerning Doubt # 3, Our Community User asked a similar Q in [3]. I haven't reviewed this Use-Case internally around Hit/Miss Ratio in the UI to answer the same. Henceforth, I shall let our fellow HBase Engineers to answer [3], which may answer your Q3 as well. Barring Q3, Let me know if your first 2 queries are addressed by [1] & [2]. Regards, Smarak [1] https://community.cloudera.com/t5/Support-Questions/Is-it-possible-for-inconsistent-read-in-Hbase-with-Memcache/m-p/359452#M238123 [2] https://nag-9-s.gitbook.io/hbase/hbase-architecture/hbase-read-merge [3] https://community.cloudera.com/t5/Support-Questions/Create-cache-miss-scenario-in-HBase-with-HDP-2-6-5/m-p/359795

smdas · ‎12-25-2022

Hello @Serhii This is an Old Post, yet I am answering the same as there are few changes with CDP recent release & ensuring Community awareness. With CDP v7.1.6 [1] allows Accumulo to be installed via Cloudera Manager. The Installation is documented via [1] & requires a Separate Parcel to be installed before attempting to add Accumulo via Cloudera Manager. Having said that, Feel free to engage with Cloudera Account Team for Customer as the investment into Accumulo isn't as par with other similar counterpart to review any long term engagement with Accumulo for meeting Customer's Use-Case. Regards, Smarak [1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/opdb-accumulo-installation/topics/opdb-accumulo-install.html

smdas · ‎12-23-2022

Hello @Girija Internally, I wasn't able to replicate the issue being faced by you. I was able to create the ConfigSet using "_default" ConfigSet as baseConfig. I am assuming the issue is specific to your Environment & your team should use the CLI to better diagnose such issue. Your team can use the below solrctl command to create a ConfigSet with Solr KeyTab: solrctl config --create Test_Config _default -p configSetProp.immutable=false Assuming the above Command fails, Running the solrctl command with "--trace" after "solrctl" & before "config" would print the trace logging & assist in troubleshooting the issue faced by your team. Regards, Smarak

smdas · ‎12-21-2022

Hello @SagarCapG Confirmed that Phoenix v5.1.0 has the Fix for "!primarykeys" to show the Primary Key linked with a Phoenix Table. Upon checking our Product Documentation, CDP v7.1.6 introduces Phoenix v5.1.0 [1]. As such, I am surprised your Team has Phoenix v5.0.0 with CDP v7.1.7, wherein Official v7.1.7 Doc [2] says Phoenix v5.1.1.7.1.7.0-551 is used. Since the Issue is fixed in Phoenix v5.1.x & CDP v7.1.6 onwards ship Phoenix v5.1.x, Kindly engage Cloudera Support to allow Support to review your Cluster for identifying the reasoning for CDP v7.1.7 using Phoenix v5.0.0. Or, Upgrade to Phoenix v5.1.x (If Customer is managing Phoenix outside of CDP) to use "!primarykeys" functionality. Regards, Smarak [1] What's New in Apache Phoenix | CDP Private Cloud (cloudera.com) [2] Cloudera Runtime component versions | CDP Private Cloud

smdas · ‎12-20-2022

Hello @brajeshreddy Since the Issue isn't replicated with CML release internally & your team have engaged Cloudera Support for further assistance, We shall Close the Post now. For our fellow Community Users, the Steps to perform the Team's Name Modification is shared above. Assuming the same isn't working for Customers, Ensure you are connected as MLAdmin & Caching is ruled out as well. Regards, Smarak

smdas · ‎12-20-2022

Hello @sekhar1 Since we haven't heard back from your side concerning the Post, We shall mark the Post as Resolved with the Action Plan to review if Traffic from your Machine isn't allowed to or from the Security Group linked with the VPC wherein the CML Workspace instances are deployed. Check if the Kubernetes Pods associated with the CML Workspace Kubernetes Cluster are Up/Running. If Yes, Such Exception should be reviewed from a Network Standpoint only. You may reach out to your Customer's AWS/Platform Team to review the Traffic between your Machine & the VPC within which the CML Workspace is deployed. Assuming your team fixed the issue outside of any Customer's Network concerns, We would appreciate your feedback to ensure our fellow Community Users can benefit from your experience. Regards, Smarak

smdas · ‎12-20-2022

Hello @brajeshreddy Thanks for using Cloudera Community. On a CML v2.0.34-b116, I was able to rename the Team's Name via UI as shared below via [1] & [2]. Upon Clicking On Team's Name, I get the UI [3], which allows me to modify the Team's Name. Since I am using the same Version as yours, the Issue likely lies with Permission (You may review the Access granted at Environment Level), Caching (You may clear the Browser Cache) etc. If both of the above checks aren't helpful, I suggest engaging Cloudera Support to ensure a Screen-Sharing Session can be done with your team to review further. Regards, Smarak [1] Name Before Change: [2] Name After Change: [3] Name Change Provisioning:

Online	Offline
Last Visited	‎12-19-2024 01:22 AM

Member Since	‎01-16-2018 09:55 AM
Last Visited	‎12-19-2024 01:22 AM
Posts	607
Kudos received	48

Cloudera Community

Re: How to enable IAM for apache airflow

Re: Apache Airflow can not connect to mssql 2008

Re: Airflow is failing to start in cloudera with i...

Re: Can we run CDP on ECS in AWS Environment

Re: CDE CLI Date argument

Re: How is the memory allocated in CDP Machine Lea...

Re: What is Kafka Gateway and Kafka MirrorMaker wh...

Re: Seperation of hot and cold data -HBase

Re: Read merge in hbase

Re: Install Accumulo using CDH 7

Re: Unable to creating config-set in Solr version ...

Re: Unable to view phoenix table primary keys via ...

Re: Renaming an CML Team name

Re: ML workspace page does NOT open

Re: Renaming an CML Team name