Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.
alex_schaetzle
Explorer
My Accepted Solutions
Show All

Cloudbreak Workspaces: Move Clusters

by Explorer in Support Questions
‎04-05-2019 08:28 AM
‎04-05-2019 08:28 AM
Hi all, is it possible to move a cluster from a workspace to another workspace? Let's say I have created a cluster in my personal workspace and want to move it to another workspace where more people have access to. Is this possible? Otherwise I have to carefully think about which cluster to spawn in which workspace in advance because it cannot be changed later on. Thanks in advance! Alex ... View more
Labels:

Re: Custom Hostname for Cloudbreak using WebUI

by Explorer in Support Questions
‎04-05-2019 06:08 AM
‎04-05-2019 06:08 AM
OK this feature should be extended (more customization options) and added to the wizard in future Cloudbreak versions in my opinion because custom hostnames is quite common requirement. ... View more

Custom Hostname for Cloudbreak using WebUI

by Explorer in Support Questions
‎04-04-2019 09:41 AM
‎04-04-2019 09:41 AM
Hi all, is there a way to define custom hostnames when deploying a cluster with the WebUI of Cloudbreak or do we have to use the CLI to do that? The documentation just mentions the CLI: https://docs.hortonworks.com/HDPDocuments/Cloudbreak/Cloudbreak-2.9.0/advanced-cluster-options/content/cb_custom-hostnames.html Is it possible to add the corresponding customDomain entries to the request that the Cloudbreak cluster creation wizard creates at the end without having to switch to the CLI? Thanks! Alex ... View more
Labels:

Re: Yarn container limits für Spark jobs

by Explorer in Support Questions
‎08-03-2018 07:59 AM
‎08-03-2018 07:59 AM
Hi @dbompart, our Spark version is 2.2.0 (HDP 2.6.3). I think it's not an issue how we set the parameter, because we actually set it this way in spark2-defaults: spark.yarn.executor.memoryOverhead=2500 And here if to output from the log: INFO YarnAllocator: Will request 12 executor container(s), each with 4 core(s) and 10692 MB memory (including 2500 MB of overhead) So the values seem to be right and the request fits my expectations (8g executor memory + 2.5g overhead). But I'm confused why the container is not killed when exceeding 10.5 GB but 12 GB: WARN YarnAllocator: Container killed by YARN for exceeding memory limits. 13.3 GB of 12 GB physical memory used. Consider boosting spark.yarn.executor.memoryOverhead. Is there any further parameter that I'm missing here? ... View more

Yarn container limits für Spark jobs

by Explorer in Support Questions
‎08-02-2018 08:02 AM
‎08-02-2018 08:02 AM
We are getting the following error when running a PySpark job on YARN: "Container killed by YARN for exceeding memory limits. 12.3 GB of 12 GB physical memory used. Consider boosting spark.yarn.executor.memoryOverhead." My question is not so much why we get this error message but where does the value for the physical memory limit come from? In our configuration we set: spark.executor.memory=8g spark.yarn.executor.memoryOverhead=2.5g As far as I know, these are the two config values that influence the container size in YARN for a Spark job. So I would expect the physical memory limit to be 10.5 GB and not 12 GB as stated in the error message. Is there any further parameter that I'm not aware of? Thanks for clearification! Alex ... View more
Labels:

Re: YARN Exception: Could not determine the proxy ...

by Explorer in Support Questions
‎05-23-2018 10:17 AM
‎05-23-2018 10:17 AM
I was not able to fix this issue up to now. The only solution I found currently is to disable Resource Manager HA. ... View more

Re: Disable ResourceManager HA

by Explorer in Support Questions
‎04-20-2018 10:17 AM
‎04-20-2018 10:17 AM
I managed to get it running by specifying all the required options usind the right flags. The documentation still seams to use the command as it was used for configs.sh (before Ambari 2.6.0) but is not valid anymore for configs.py. The documentation should be adapted to the new command for configs.py. ... View more

Re: Disable ResourceManager HA

by Explorer in Support Questions
‎04-20-2018 10:08 AM
‎04-20-2018 10:08 AM
I already tried that but I'm getting the same error. ... View more

Disable ResourceManager HA

by Explorer in Support Questions
‎04-20-2018 08:46 AM
‎04-20-2018 08:46 AM
Hi, I'm trying to disable ResourceManager High Availablity because I cannot find a solution to a problem with proxy server redirection (YARN Exception: Could not determine the proxy server for redirection) and I want to check if the problem is gone when not using YARN HA. I'm following the documentation to do that (Disable ResourceManager High Availability) but I'm getting an error when executing the script in step 2. I call it like this: /var/lib/ambari-server/resources/scripts/configs.py get m0201.cl.psiori.com psiori yarn-site yarn-site.json The hostname of the Ambari server is "m0201.cl.psiori.com" and the cluster name is "psiori" but if fails with the error: configs.py: error: One of required options is not passed What am I doing wrong? And why can I not simply remove the corresponding configuration values mentioned in step 3 in the Ambari UI? Thx, Alex ... View more
Labels:

Re: YARN Exception: Could not determine the proxy ...

by Explorer in Support Questions
‎04-20-2018 08:34 AM
‎04-20-2018 08:34 AM
Is there nobody with the same issue? ... View more
Contact Me
Online
Offline
Last Visited
‎02-29-2020 11:21 AM
Public Statistics
Date Registered ‎02-15-2018 10:40 AM
Last Visited ‎02-29-2020 11:21 AM
Total Messages Posted 30