Member since
07-01-2015
460
Posts
78
Kudos Received
43
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1360 | 11-26-2019 11:47 PM | |
1309 | 11-25-2019 11:44 AM | |
9517 | 08-07-2019 12:48 AM | |
2193 | 04-17-2019 03:09 AM | |
3520 | 02-18-2019 12:23 AM |
08-03-2018
12:29 AM
1 Kudo
Glad you got it working. Just to note something like this did work for me for hvm instance types: hvm=c5.large,\ c5.xlarge,\ c5.2xlarge,\ c5d.2xlarge There should be a ec2.virtualizationmappings.properties.example file you can look at, modify as an example.
... View more
07-27-2018
10:55 AM
I am seeing similar issue with ServiceMonitor and Host monitor when using Redhat 6.8 (Santiago) CM/CDH is 5.11.1 After adding JAVA_TOOL_OPTIONS=-Xss2m to hostmonitor and service monitor configuration is works fine. Is this a known issue with Redhat 6.7 as well ? (The link you mentioned is centos and its 6.9)
... View more
07-05-2018
10:57 AM
You can find Cloudera Management Service properties here: https://www.cloudera.com/documentation/enterprise/properties/5-15-x/topics/cm_props_mgmtservice.html
... View more
07-03-2018
07:29 AM
1 Kudo
It could be a memory issue, and it does not relate to the condition of the server at all. The containers running on YARN have their max memory size, and the NodeManager carefully watch over. When the container allocates more memory, then it gets killed. Also keep in mind that the container size is the whole JVM, so if it is not 100% for map or reduce data. For example on Spark you have to calculate how big container do you need if you want to use X MB of memory for cache and Y MB of memory for code.
... View more
07-02-2018
06:02 AM
Hi Tomas79, I think a Python script will work if you start it with a shebang (e.g., #/usr/bin/env python). Director uses essentially the following shell command to run a script, so I can see a Python script working with it. sudo chmod 700 'scriptName' && sudo 'scriptName' Even so, running a Python script directly is a little hacky because we haven't thought about Director running anything other than shell scripts with this capability, so I can't guarantee that this command won't change in the future to not work with a Python script. However, something like the following shell script will be fine. #!/usr/bin/env bash
python <<EOF
print 'Hello world!'
EOF Or: #!/usr/bin/env bash
curl -O http://host/path/to/downloadable/script.py
python script.py Or rather than downloading the script, bake it into your AMI / image. So, there are a few options.
... View more
06-25-2018
12:58 AM
Hi, is this the only solution for this kind of error - increasing the memory limit? Thannks
... View more
06-19-2018
02:00 AM
1 Kudo
You did not mentioned the version of CDH. But I think the problem is that spark launches many executors to read, and those executors are not co-located with the Kudu tablet servers. I dont know if you are just reading/filtering the data, or reading and writing into parquet - it depends how the spark job is executed. What I also noticed, that running multiple spark jobs agains the same table (with different partitions) did not help either.
... View more
06-04-2018
10:54 AM
1 Kudo
Applying patch - i.e. upgrading to 5.13.3 solved that problem
... View more
01-31-2018
10:05 AM
1 Kudo
It looks like your screenshot is of the "scans" dashboard on the web UI. This dashboard shows counters for a single scan, and a single scan would only come from a single task, not aggregate across them. I am guessing you're hitting KUDU-2231, a performance bug recently fixed. The bug fix appears in CDH 5.14.0. Since this is a performance issue that is not a regression and does not affect correctness, we have not yet backported to any prior releases. -Todd
... View more