About vaishaakb

vaishaakb · ‎01-23-2023

What processing time should we expect? It is unfortunate that I am not in a position to share an ETA on this as this certification involves larger teams within Cloudera. We are considering which CentOS version to use for the upcoming installation and we are unsure if it is worth waiting. Sure, Is there a specific Business goal that aligns with making use of CentOS 9 Stream? Else, May I suggest you try the CDP 7.1.7 SP1 on CentOS 8.6? HTH.

vaishaakb · ‎01-22-2023

@DrManu Hey Manuel, G'Day! I appreciate your patience with me on this ask. I got a chance to notify this to our Product Team by flagging an Internal JIRA, as follows: INTERNAL JIRA Details: JIRA ID: PLATFORM-197 JIRA Subject: Add Support for CentOS 9 Stream Once the Platform(CentOS 9 Stream) is certified(Tested) by our Product Team, the same will be updated in the Product Support Matrix[0a]. [0a] https://supportmatrix.cloudera.com/ I hope this helps. If you find my answer addressed your ask, please take a minute to login and click the "Accept as Solution" button, so this will be highlighted for this thread. - V

vaishaakb · ‎01-19-2023

Hello @DrManu Thank you for bringing this in our Community. Per our Products' Support Matrix[0a], We have tested our CDP Private Cloud lineup until CentOS 8.6 [0a] https://supportmatrix.cloudera.com/ Are there any plans to officially support it in the near future? I'll start an internal conversation to keep track of this request. You can expect me to update this thread in some time. Additionally, Do you have specific CDP version in mind that you'd like to tag this request to? Let me know your thoughts. - V

vaishaakb · ‎10-18-2022

Thanks for sharing how that was resolved. Did we achieve the end-goal? Also, Q. Did you get to try/test the steps mentioned in our Blog, comparing your spark-submit?[0] [0] https://blog.cloudera.com/introducing-apache-spark-on-docker-on-top-of-apache-yarn-with-cdp-datacent... In the above blog, Check out the Demo I section, " Running PySpark on the gateway machine with Dockerized Executors in a Kerberized cluster. " V

vaishaakb · ‎10-17-2022

Hey @prabhu111 Thank you for bringing this to our community. >>>> [Cloudera][DriverSupport] (1100) SSL certificate verification failed because the certificate is missing or incorrect. <<<< Since I could not narrow-down what could be the probable cause(s) for this. Here is something I would suggest you start with: 1. Did the problem occur after the SSL Certs were renewed/updated on this Cluster? 2. What is the ODBC Connection string used to connect to Impala? SANITY CHECKS: 3. Try enabling TRACE level ODBC DRIVER logging and observe the logs for probable cause. The instructions are in Page 19 of this Driver doc: https://www.cloudera.com/documentation/other/connectors/impala-odbc/latest/Cloudera-ODBC-Driver-for-Impala-Install-Guide.pdf Assuming that you are using "cacert.pem" under "C:\Program Files\Cloudera-ODBC Driver-for-Impala\lib" for "Trusted Certificates". The trusted CA certificates in the cacert.pem file will be used to verify the Impala server. 4. You can also verify the certs with the following openssl command: openssl s_client -showcerts -servername <server fqdn> -connect <server>:21050 5. See if you have Valid/Proper cacert.pem file with the non-expired dates? If it is expired it needs to be updated/renewed with the new CA certs. HTH. V

vaishaakb · ‎10-17-2022

Hello @vtpcnk Good Day! Circling back to see if my post helped you with this task. If yes, Please mark the answer as "Accept as Solution". So this will be highlighted as solution to this ask for other users in our community. V

vaishaakb · ‎10-17-2022

Hey @hanumanth Thank you for marking this post as Accepted Solution. If you find my sanity checks helpful, you can also thank me by clicking on the thumbs up button:) V

vaishaakb · ‎10-17-2022

Hey Pete! I appreciate your patience! Sorry, I was away. > Could I impose on you further and and ask how I would do it to individual hosts / roleids? Sure, I will be happy to help you with that. > how I would do it to individual hosts / roleids? For this, You would want to get the hostids from CM-API Example: $ curl -u <admin-user>:<admin-password> http://<CM_SERVER_FQDN>:7180/api/v44/hosts Followed by the above, You can try entering replacing the hostId in the items section[0] [0] https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/json_ApiRoleList.html Let me know if this helps? V

vaishaakb · ‎10-17-2022

Hey @fares_ Sorry about the delayed update. I was away. Q. how was the Docker Mount issue resolved? Please share the context of the steps taken. >>> I solved the mount issue, but that took me back to the same main issue which mentioned in the main post. Are you still observing invalid Docker mount error? Did you get to try/test the steps mentioned in our Blog?[0] [0] https://blog.cloudera.com/introducing-apache-spark-on-docker-on-top-of-apache-yarn-with-cdp-datacenter-release/ In the above blog, Check out the Demo I section, " Running PySpark on the gateway machine with Dockerized Executors in a Kerberized cluster. " Keep us posted. V

vaishaakb · ‎10-17-2022

Hello @TalhaKilic Did my response help you with the request made? If yes, Please mark my reply as 'Accept As Solution' of if you still need further help, Please update this thread. V

vaishaakb · ‎10-04-2022

Hey there @TalhaKilic Thank you for posting this in our Cloudera Community. Is there a specific Business use-case that you are looking for? If yes, Please share your high-level use-case. Else, I presume this question is to check compatibility of CDP on Google Anthos and Here is my finding: Does CDP supported Google Anthos? It is unfortunate that CDP is not yet tested on Google Anthos. However, we do have a currently 'OPEN' internal JIRA flagged for the same with an ID, PVCRB-122 in which CDP will be tested by our Larger team. Let me know if this helps:) V

vaishaakb · ‎10-04-2022

Hey @hanumanth Circling back to know if the posted checks helped you with this task. Additionally, Please share what was done to sort this situation. HTH!

vaishaakb · ‎09-30-2022

[30/Sep/2022 19:37:38 +0000] 1048 MainThread agent WARNING Long HB processing time: 7.00362992287 [30/Sep/2022 19:38:01 +0000] 1048 DnsResolutionMonitor throttling_logger INFO DnsTest not running. Java not located. Hey @hanu Thank you for writing this in our Community. SANITY CHECKS: The above states that there could be no JAVA_HOME set. Q. Can you review if you this host has right JAVA_HOME set? or configured with a custom location? 0a. You can navigate to CM > Hosts > All Hosts> Affected Host > Quick Link > Host agent and check the value for JAVA_HOME inside the section "Component Info" 1. Make sure you have the proper values for CM agent under /etc/cloudera-scm-agent/config.ini? 1a. Compare and fix the UUID of the CM agent on the affected node, if need be. Compare CM > Hosts > All Hosts > Affected Host UUID with /var/lib/cloudera-scm-agent/uuid If they do not match, Carry on with the following steps: Navigate to CM > Hosts > All Hosts 1. Identify the UUID Click on Affected Host > Under “Details” copy the “Host ID”. This is the affected CM agent's UUID. 2. SSH to affected host, and Stop the CM agent #systemctl stop cloudera-scm-agent 3. Navigate to var/lib/cloudera-scm-agent/ 4. Make a backup of the existing uuid file. # cp var/lib/cloudera-scm-agent/uuid /tmp/agent-uuid.bk 5. Change the contents of the UUID file with the CM-UI uuid (i.e from step 1) # echo -n “<UUID_from_CM-UI>” > /var/lib/cloudera-scm-agent/uuid 6. Start the Agent: # systemctl start cloudera-scm-agent 2. Check Network configuration if there is no Heartbeat to CM Server. Try comparing and update the following as needed to match one of the nodes that does not have any issues. /etc/cloudera-scm-agent/config.ini /etc/resolv.conf /etc/hosts /etc/nsswitch.conf Once the above is updated/fixed, the CM agent requires restart: # systemctl restart cloudera-scm-agent Let us know if this helps, else, Share your observations. Thanks!

vaishaakb · ‎09-19-2022

Hey There @fares_, Thank you for writing this in our community. There was a similar situation with this User and see if this is related: https://community.cloudera.com/t5/Support-Questions/Container-exited-with-a-non-zero-exit-code-13-Error-file/m-p/349268/highlight/true#M235584 Additionally, I could see the Error code being 1 in the shared snip of the log, I was able to trace back the SparkExitCodes[0] definition for you to co-relate(To triangulate the root cause): https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/util/SparkExitCode.scala And Finally, Did you get to go through our similar Blog against your test-case? https://blog.cloudera.com/introducing-apache-spark-on-docker-on-top-of-apache-yarn-with-cdp-datacenter-release/ Keep us posted on how it goes.

vaishaakb · ‎09-19-2022

Hey @spotxsysops Thank you for bringing this to our community. You may want to test this internally with this PUT call: to configure DataNode Data Directory / dfs.datanode.data.dir and NodeManager Local Directories / yarn.nodemanager.local-dirs Q1. To configure DataNode Data Directory / dfs.datanode.data.dir PUT /api/v31/clusters/Cluster%201/services/HDFS-1/roleConfigGroups/HDFS-1-DATANODE-BASE/config { "items": [ { "name": "dfs_data_dir_list", "value": "/path/to/dir" } ] } Q2. To configure NodeManager Local Directories / yarn.nodemanager.local-dirs PUT /api/v31/clusters/Cluster%201/services/YARN-1/roleConfigGroups/YARN-1-NODEMANAGER-BASE/config { "items": [ { "name": "yarn_nodemanager_local_dirs", "value": "/path/to/dir" } ] } However, The Generic CM API End-Points for RoleConfigGroups can be found here: https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/resource_RoleConfigGroupsResource.html Let me know if this helps?

vaishaakb · ‎09-19-2022

Hello @poorva Did you get a chance to review the Failed Application logs from Resource Manager UI? You can also share the exception from the same time frame so we can help you further with a plan of action. Let us know your thoughts.

vaishaakb · ‎09-16-2022

Hey @abdebja , Thank you for writing this in our community. >>> What configuration item could fix this in cloudera management services? As mentioned in the KB article above reply, the root cause of this issue is highly suspected to be JVM Pauses on the Reports Manager, which is a part of Cloudera Management Services. Did you get to review/try the changes mentioned in the above KB article? As a Best practice, I am also referring a Public Documentation https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/installation/topics/cdpdc-reports-manager.html Let me know if this helps.

vaishaakb · ‎04-09-2021

Hi @bsaad Thank you for helping me with the logs, Sounds like a "Hardening" issue, if that is what to be mentioned by VirtualBox Community. Please read these and triage the situation based on your System Specs: VirtualBox Threads: https://forums.virtualbox.org/viewtopic.php?f=6&t=95559 Diagnosing VirtualBox Hardening Issues Hope this helps! Is your question answered? If Yes, Feel free to mark the answer as "Accept as Solution". If you find this reply helpful, say thanks by clicking on the "thumbs-up" button.

vaishaakb · ‎04-09-2021

Hey @bsaad Thank you for keeping us posted. Glad that you were able to figure it out. >>> However, I am still not able to run cloudera. What is it that you are observing? Or Steps that you take? What about the logs? Did you get to check the logs in the specified location? Let me know how it goes. - Vaishaak

vaishaakb · ‎04-08-2021

Hi @bsaad Thank you for posting in Cloudera Community. Did you get to look at the logs for more details on the situation?: C:\Users\badao\VirtualBox VMs\cloudera-quickstart-vm-5.4.2-0-virtualbox\Logs\VBoxHardening.log Also, I came across these from Microsoft and VirtualBox Community that may be resourceful too. Microsoft: https://answers.microsoft.com/en-us/windows/forum/all/how-to-fix-virtualbox-failed-to-open-a-session-for/39b7cb91-8983-46f4-897c-99a4709dd5c4 VirtualBox Community: https://forums.virtualbox.org/viewtopic.php?t=33196 Hope this helps! Is your question answered? If Yes, Feel free to mark the answer as "Accept as Solution". If you find a reply helpful, say thanks by clicking on the "thumbs-up" button.

vaishaakb · ‎04-07-2021

Hi, @vtpcnk Greetings! Thank you for posting in Cloudera Community. You can use a RegEx pattern like this, Try this out: CM > Backup > Schedules > Actions > Edit Configurations > Advanced > Path Exclusion ^(?!.*.tmp).*$ For more information, here are some nicer reads: Section 7: CLDR Doc (Adding Exclusion) How-to: Building/Testing RegEx Hope this helps! Is your question answered? If Yes, Feel free to mark the answer as "Accept as Solution". If you find a reply helpful, say thanks by clicking on the "thumbs-up" button.

vaishaakb · ‎04-06-2021

Hello @meenzoon, I am glad to hear that this situation is resolved and really appreciate you providing the steps you took to solve this. Have a good one!

vaishaakb · ‎04-05-2021

Hey, @meenzoon Hope this message finds you. I did get a chance to review your logs: 1. Was there a recent upgrade from CDH5 to CDH6? 2. Are there any other roles in this instance not working? 2021-03-30 16:37:45,512 INFO exited: 178-hue-HUE_SERVER (exit status 0; not expected) 2021-03-30 16:37:46,174 INFO gave up: 178-hue-HUE_SERVER entered FATAL state, too many start retries too quickly The above indicates there seems to be an issue with Supervisord. However, 3. What do you observe in the Hue role's stderr and stdout? Hint: You can get the Hue Process's stderr and stdout under CM > Hue > Instances > Hue Server > Log Files > Stdout and Stderr(Quick-link) Let us know how it goes.

vaishaakb · ‎01-08-2020

Hi Snehasish, From your post, I see that you are referring to Please refer these links for more information on S3 BDR: >> Our Blog on S3 and BDR https://blog.cloudera.com/using-amazon-s3-with-cloudera-bdr/ >> Hive Replication to/from Cloud storage https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_bdr_hive_replication.html#concept_ld2_v2z_qm "From the documentation, If you configure replication of a Hive table and then later drop that table, the table remains on the destination cluster. The table is not dropped when subsequent replications occur." To answer your query, That depends on How the delete policy of the Replication Job is set to and also the Permission attributes set in the Advanced configuration of the replication schedule, Refer to the "Hive Replication to/from Cloud storage" on both the attributes for better understanding and write back to us if you need further clarification. Regards V

vaishaakb · ‎01-07-2020

Hi @Nick Error found before invoking supervisord: [Errno 2] No such file or directory: '/sys/fs/cgroup/cpuacct,cpu/system.slice/docker-542551d4e6b6ca6c6ab65a6f3b51c538b126a1762b36b35b7ba9023ed262f953.scope/185-yarn-RESOURCEMANAGER' The above looks like you have this inside a Docker container. Please confirm. - Could you do the following to help us understand this better 1. Have you tried running the docker as the privileged mode? if you do not have mounts configured? #docker run -t -i --privileged os-flavor:os-version (example centos:6) 2. Create directory using root where the CM is trying to create the specified path: #mkdir /sys/fs/cgroup/cpuacct,cpu/systemd/system.slice/ Let me know how it goes. Thanks V

vaishaakb · ‎01-07-2020

Hello @kotni, The Stack trace says the connection is refused on the NameNode port(9870) dfs.namenode.http-address , hence make sure the required ports are configured/opened in both the source and Destination clusters on performing Distcp. - Here is the list of ports used by CDH components and DistCP - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cdh_ports.html - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/install_ports_distcp.html Post that, Try out the below: - Examples of DistCP syntaxes to try out https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cdh_admin_distcp_data_cluster_migrate.html#distcp_examples Let us know how it goes. - Vaishaak

vaishaakb · ‎05-10-2019

Hello TheRabroub, Thank you for posting this in our Community. E: Failed to fetch https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/pool/contrib/o/oracle-j2sdk1.7/oracle-j2sdk1... transfer closed with 118225812 bytes remaining to read "The above could mean that the connectivity between the Host and the Cloudera archive might have been intervened by network congestion/flaky network" E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? END (100) remote package oracle-j2sdk1.7 Have you tried the above-mentioned? If not, Since you are on Ubuntu, Try to fix/retry the missing packages by executing this: #sudo apt-get update or #sudo apt-get update --fix-missing Let us know how it goes. Cheers

Date Registered	‎10-18-2018 04:08 AM
Last Visited
Total Messages Posted	74
Total Kudos Received	4

Re: CDP Private Cloud on CentOS 9

Re: cloudera agent has issue after OS RHEL 7.5 to ...

Re: REPORTSMANAGER multiple hprof files are gettin...

Re: Does not start HUE_SERVER

Re: CDP Private Cloud on CentOS 9

Re: CDP Private Cloud on CentOS 9

Re: CDP Private Cloud on CentOS 9

Re: Python script with docker on yarn results in j...

Re: [Cloudera][DriverSupport] (1100) SSL certifica...

Re: BDR - HDFS - exclusion

Re: cloudera agent has issue after OS RHEL 7.5 to ...

Re: Using api to configure datanode directories

Re: Python script with docker on yarn results in j...

Re: Does CDP supported Google Anthos ?

Re: Does CDP supported Google Anthos ?

Re: cloudera agent has issue after OS RHEL 7.5 to ...

Re: cloudera agent has issue after OS RHEL 7.5 to ...

Re: Python script with docker on yarn results in j...

Re: Using api to configure datanode directories

Re: SparkContext was shut down

Re: REPORTSMANAGER multiple hprof files are gettin...

Re: Please help: BIOS Virtualisation enabled but c...

Re: Please help: BIOS Virtualisation enabled but c...

Re: Please help: BIOS Virtualisation enabled but c...

Re: BDR - HDFS - exclusion

Re: Does not start HUE_SERVER

Re: Does not start HUE_SERVER

Re: Hive Replication to S3

Re: Cgroups Configuration on Cloudera

Re: hadoop distcp failing with Invalid arguments: ...

Re: Cloudera 3 Nodes installation failed