Member since
11-12-2013
41
Posts
11
Kudos Received
7
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4450 | 10-18-2019 02:11 PM | |
4725 | 07-10-2019 03:20 PM | |
3442 | 03-24-2019 02:52 PM | |
5299 | 03-20-2019 09:01 AM | |
1700 | 12-13-2018 05:06 PM |
03-23-2019
04:30 PM
Before we get into metrics and other lower-level troubleshooting techniques, let's start with how you're reading. What are you using to read? If it's the raw Kudu API, are you using the LEADER_ONLY replica selection policy? If so, and if your three node cluster is heavily skewed so that the majority of leader replicas are on one node, it's possible for that node to be servicing the majority of your scans.
... View more
03-20-2019
09:01 AM
1 Kudo
Indeed, there's going to be a significant amount of memory consumed just as overhead to support that number of tablets. So you should either reduce the number of tablets per tserver, or increase the amount of RAM available to Kudu on those heavily-loaded machines.
... View more
03-19-2019
11:24 AM
That is interesting. How many tablets are hosted by this tablet server?
... View more
12-13-2018
05:06 PM
1 Kudo
Yes, unfortunately this was an oversight, and will be corrected in a future release. For now you can either do the workaround you suggested (manually underallocating for the others, then manually configuring Kudu's memory limit). Or, if you want to roll up your sleeves a bit: ssh to the CM server machine. For each file named KUDU*.jar (in /usr/share/cmf/common_jars, I believe): Extract the JAR file using the 'jar' utility. Modify descriptor/service.sdl by finding the memory_limit_hard_bytes entry, adding a comma after "default" : 4294967296, and adding a new line with the contents "autoConfigShare" : 100 Recreate the JAR around the extracted files (including the modified service.sdl) Overwrite the existing KUDU*.jar files with the new ones you created in step #5. Restart the CM server. You should now see an entry in he static service pools UI for Kudu's Tablet Server Hard Memory Limit.
... View more
08-30-2018
10:29 AM
1 Kudo
Hey Paulo, I saw you posted the same question to the Apache Kudu users mailing list. Here's the response I wrote there: Check out this older post by Todd Lipcon about the 300 column limit:
http://mail-archives.apache.org/mod_mbox/kudu-user/201706.mbox/%3CCADY20s7iT7%2BrVZNhagnNFUjk7-nNMxJK6%2BnHV%2B2SzpHXKFxvmw%40mail.gmail.com%3E
There are probably other folks who run with over 300 columns in their
schemas, but it's not something Kudu developers are actively testing.
You're certainly welcome to try it, and I'm pleased that your 400
column schema is working well for you, but just be aware that you may
run into unforeseen issues.
... View more
08-03-2018
12:05 PM
2 Kudos
This error indicates that a Kudu tablet server has closed a scanner because it hadn't been accessed in some time. It could manifest on a particularly slow query that very infrequently fetches new data from Kudu. Can you share the Impala query profile for one of the failed queries, as well as the Impala daemon coordinator log? You can also work around the issue by reconfiguring Kudu's --scanner_ttl_ms flag to a much higher value (the default is 60s), though this will come at the potential cost of Kudu memory if any clients are orphaning their scanners.
... View more
01-03-2018
01:07 PM
2 Kudos
Quick note: Kudu calls the "set of CFiles which contain the base data and the delta data" a DiskRowSet. But your understanding is correct: during a scan, the contents of the MemRowSet and some DiskRowSets are scanned for data. During a tablet copy, both the WAL segments and the CFiles are copied.
... View more
01-02-2018
02:16 PM
NTP synchronization (and specifically enforcing a maximum clock error on each node) helps guarantee Kudu's transaction semantics. This page has more details, as does the Kudu design paper. The maximum clock error is an NTP concept, and refers to the upper bound of time that the local machine's clock may deviate from whatever NTP clock it's synchronized with. Machines with well-configured NTP installations should guarantee some sort of stable maximum clock error that you can use for Kudu's max_clock_sync_error_usec configuration flag. Unfortunately I don't understand Kudu transactions well enough to explain its effect on Kudu transaction semantics.
... View more
01-02-2018
01:56 PM
I'm not Andrew but I don't understand your question. Tablet replicas are flushed and/or compacted independently of one another, which means the physical layout of each one may be different; thus it would be incorrect to consider them "physically replicated". Furthermore, all scans (whether CLOSEST_REPLICA or otherwise) are read-only operations and thus don't trigger WAL garbage collection or any other kind of read-write server-side action.
... View more
11-12-2013
02:38 PM
This is a known issue with CM and MySQL 5.6.7 or newer. We're working on it and should have it fixed in an upcoming release. In general, MySQL 5.6 isn't yet supported, so the safest thing you can do is to downgrade to an older version, like 5.5. Switching to MyISAM is a really bad idea; a bunch of stuff won't work as expected down the line.
... View more
- « Previous
-
- 1
- 2
- Next »