Member since
12-13-2013
39
Posts
8
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5787 | 10-10-2017 08:05 PM |
07-29-2019
05:12 PM
I filed https://issues.apache.org/jira/browse/IMPALA-8807 to fix the docs.
... View more
04-18-2019
03:36 PM
@Braz wrote: Is there not a Kudu command which will allow for obtaining table size information? If not, then how does Cloudera Manager perform this? We would like to be able to replicate this behavior so that we can configure e-mail alerts to be sent whenever a table reaches a particular size. Thanks, Braz CM is scrapping and aggregating the /metrics pages from the tablet server instances for each tablet/table. Have you reviewed CM triggers/alerts?[1] You might be able to configure email alerts with a similar trigger rule for table sizes. Alternatively, you could implement what CM currently does by scraping each tablet server's /metrics page and aggregating the data together per tablet/table. [1] https://www.cloudera.com/documentation/enterprise/latest/topics/cm_dg_triggers_usecases.html
... View more
03-24-2019
09:57 AM
1 Kudo
Impala scanners internally have a RowBatch queue that allows Impala to decouple I/O from CPU processing. The I/O threads read data into RowBatches and put them into a queue, CPU threads asynchronously fetch data from the queue and process them. RowBatchQueueGetWaitTime is the amount of time CPU threads wait on data to arrive into the queue. Essentially, it means the CPU threads were waiting a long time for the I/O threads to read the data.
... View more
12-13-2018
05:06 PM
1 Kudo
Yes, unfortunately this was an oversight, and will be corrected in a future release. For now you can either do the workaround you suggested (manually underallocating for the others, then manually configuring Kudu's memory limit). Or, if you want to roll up your sleeves a bit: ssh to the CM server machine. For each file named KUDU*.jar (in /usr/share/cmf/common_jars, I believe): Extract the JAR file using the 'jar' utility. Modify descriptor/service.sdl by finding the memory_limit_hard_bytes entry, adding a comma after "default" : 4294967296, and adding a new line with the contents "autoConfigShare" : 100 Recreate the JAR around the extracted files (including the modified service.sdl) Overwrite the existing KUDU*.jar files with the new ones you created in step #5. Restart the CM server. You should now see an entry in he static service pools UI for Kudu's Tablet Server Hard Memory Limit.
... View more
06-12-2018
05:12 PM
@mauriciothat's great news! Thanks for the update. We do need to get this documented though.
... View more
03-27-2018
01:57 PM
Thanks very much Tim for looking up the JIRA. Yikes, been open since 2014. As John pointed out there, column order info must be in the metastore since hive show create table displays fine, so seems like this should be a simple change to how impala reads that info. Upvoted the JIRA.
... View more
01-02-2018
12:02 AM
It's stuck in a CREATED state (if I remember correctly at one time I could get to the daemon page), I don't remember about other queries cause when I've checked only this query was on the daemon and the daemon was page stuck in my browser. My daemon has exactly 16GB. Thanks, I'll try that, there any other tracing tool I can use to check the daemon or the query?
... View more
12-31-2017
11:33 PM
Hi, Ok, we will check that. Mauricio, thanks for the detailed answer!
... View more
11-14-2017
12:17 PM
Hi mauricio, Impala currently does not support graceful node decommissioning. We're tracking work on this feature in IMPALA-1760, but we currently are not targeting it for a particular release. Unfortunately that only leaves the option of killing the daemon. Cheers, Lars
... View more
10-10-2017
08:10 PM
Another option I forgot to mention: if your table is partitioned, and your insert query uses dynamic partitioning, it will generate 1 file per partition: insert into table2 partition(par1,par2) select col1, col2 .. colN, par1, par2 from table1; ... again up to the max parquet file size currently set, so you can play with that max to achieve 2 files per partition. https://www.cloudera.com/documentation/enterprise/5-9-x/topics/impala_partitioning.html#partition_static_dynamic
... View more