11-11-2018 11:18 PM
we have a very very slow DML Metastore update on simple insert queries into HDFS table (select constant queries). I am not sure if we are hitting https://issues.apache.org/jira/browse/IMPALA-1480, because the table is not partitioned, however I suspect that the number of files under the table can cause the issue (100k+)
After rebuilding the table the DML queries are running fine, is this a known limitation of Impala or a bug?
---------------- Max Per-Host Resource Reservation: Memory=0B Per-Host Resource Estimates: Memory=10.00MB Codegen disabled by planner F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=48B mem-reservation=0B WRITE TO HDFS [base.import_processed, OVERWRITE=false] | partitions=1 | mem-estimate=48B mem-reservation=0B | 00:UNION constant-operands=1 mem-estimate=0B mem-reservation=0B tuple-ids=0 row-size=48B cardinality=1 ----------------
11-19-2018 03:20 AM