Member since
07-29-2015
535
Posts
141
Kudos Received
103
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 8901 | 12-18-2020 01:46 PM | |
| 5898 | 12-16-2020 12:11 PM | |
| 4642 | 12-07-2020 01:47 PM | |
| 2797 | 12-07-2020 09:21 AM | |
| 1929 | 10-14-2020 11:15 AM |
10-23-2019
03:55 PM
Thanks for your reply, but it didn't work. I tried using other functions but it brings up the date time without milliseconds. But I want the milliseconds as well.
... View more
09-20-2019
10:04 AM
@Zane- I'm late but can provide some additional insight. I think the suggestion in the error message is a good one (I'm biased because I wrote it, but some thought went into it). "Memory is likely oversubscribed. Reducing query concurrency or configuring admission control may help avoid this error". The general solution for this is to set up admission control with some memory limits so that memory doesn't get oversubscribed, and so that one query can't gobble up more memory than you like. I did a talk at strata that gave pointers on a lot of this things - https://conferences.oreilly.com/strata/strata-ca-2019/public/schedule/detail/73000 In this case you can actually see that query 2f4b5cff11212907:886aa1400000000 is using Total=78.60 GB memory, so that's likely your problem. Impala's resource management is totally permissive out of the box and will happily let queries use up all the resources in the system like this. I didn't see what version you're running, but there were a lot of improvements in this area (config options, OOM-avoidance, diagnostics) in CDH6.1+ There's various other angles you can take to improve this - if the queries using lots of memory are suboptimal, tuning them (maybe just computing stats) makes a big difference. You can also
... View more
07-29-2019
05:12 PM
I filed https://issues.apache.org/jira/browse/IMPALA-8807 to fix the docs.
... View more
07-15-2019
01:52 PM
Probably, the version we are using is 5.12.
... View more
06-18-2019
03:30 PM
Thanks a lot, Tim!
... View more
06-14-2019
11:49 PM
Thanks for your quick reply.
... View more
06-13-2019
10:42 AM
Yeah I agree there is some inconsistency in behaviour here - the casting rules, especially around NULL, are too complex and inconsistent.
... View more
04-18-2019
10:09 AM
2 Kudos
If you are mainly accessing the table using Impala, I'd recommend Impala's compute stats for best performance of Impala. There are some subtle differences in the stats collected (whether they're partition or table-level). The engines can interoperate but Impala can generally generate better plans with the full set of stats from "COMPUTE STATS"
... View more
04-17-2019
10:28 PM
Thank you very much Tim. Provided link has clarified my doubt.
... View more
04-17-2019
06:00 PM
1 Kudo
In it's default configuration, metadata is cached until an "INVALIDATE METADATA" command evicts the table from the cache. Or until the catalog is restarted. In 5.16 and 6.1+ there are some non-default options that will evict metadata after a particular timeout. At some point these will become the defaults. Table stats are collected and stored in the hive metastore when you run a "compute stats" command. They are then just part of the table metadata.
... View more