There's a 64kb limit on strings in Kudu but otherwise you can store any binary data in them.   https://docs.cloudera.com/documentation/kudu/5-10-x/topics/kudu_known_issues.html#schema_design_limitations ... View more

Upgrading to a newer version of Impala will solve most scalability issues that you'd see on Impala 2.9, mostly because of https://blog.cloudera.com/scalability-improvement-of-apache-impala-2-12-0-in-cdh-5-15-0/. ... View more

We have some background on schema evolution in Parquet in the docs - https://docs.cloudera.com/runtime/7.2.2/impala-reference/topics/impala-parquet.html. See "Schema Evolution for Parquet Tables". Some of the details are specific to Impala but the concepts are the same across engines including Hive and Spark that use parquet tables. At a high level, you can think of the data files being immutable while the table schema evolves. If you add a new column at the end of the table, for example, that updates the table schema but leaves the parquet files unchanged. When the table is queried, the table schema and parquet file schema are reconciled and the new column's values will be all NULL.   If you want to modify the existing rows and include new non-NULL values, that would require rewriting the data, e.g. with an INSERT OVERWRITE statement for a partition or a CREATE TABLE .. AS SELECT to create an entirely new table. Keep in mind that traditional Parquet tables are not optimized for workloads with updates - Apache Kudu in particular and also transactional tables in Hive3+ have support for row-level updates that is more convenient/efficient.   We definitely don't require rewriting the whole table every time you want to add a column, that would be impractical for large tables! ... View more

"... or because memory is running low or because the extra scanner threads are not needed to keep up with the consumer from the scan node." is how I should've finished that. Not sure what happened there, I did hit the main points but truncated the last sentence. That part of Impala has had a lot of improvements for performance and observability (i.e. more info in the profile) since CDH5.10, FWIW, I'd guess on a later version this wouldn't be a problem or would be easier to debug at least. ... View more

One difference is how fast it's reading from disk - i.e. TotalRawHdfsReadTime . In CDH5.12 that includes both time spend fetching metadata from the HDFS namenode and actually reading the data off disk. If you're saying that it's only slow on one node, that probably rules out HDFS namenode slowness, which is a common cause. So probably it's actually slower doing the I/O. Note: in CDH5.15 we split out the namenode RPC time into TotalRawHdfsOpenTime to make it easier to debug things like this. I don't know exactly why I/O would be slower on that one node, it might require inspecting the host to see what's happening and if there's more CPU or I/O load on that host.   We've seen that happen if a node is more heavily loaded than other nodes because of some kind of uneven data distribution. E.g. one file is very frequently accessed, maybe if there's a dimension table that is referenced in many queries. That can sometimes be addressed by setting SCHEDULE_RANDOM_REPLICA as a query hint or query option https://docs.cloudera.com/documentation/enterprise/latest/topics/impala_hints.html or https://docs.cloudera.com/documentation/enterprise/latest/topics/impala_schedule_random_replica.html. Or even by enabling HDFS caching for the problematic table (HDFS caching spreads load across all cached replicas). Another possible cause, based on that profile, is that it's competing for scanner threads with other queries running on the same node - AverageScannerThreadConcurrency is lower in the slow case. This can either be because other concurrent queries grabbed scanner threads first (there's a global soft limit of 3x # cpus per node) or because ... View more

You need to run compute stats on the base tables referenced by the views - compute stats directly on a view isn't supported. ... View more

Impala can query views. Computing table stats on tables accessed by Impala queries is necessary to get the best performance, particularly for complex queries.   That's probably not the cause of whatever your user saw, but you need to include a query profile or the query status error message at least for us to give any tips about. ... View more

Great news!   ... View more

These are good questions that come up frequently.   https://docs.cloudera.com/runtime/7.2.2/administering-kudu/topics/kudu-security-trusted-users.html discusses the issue. In summary, Hive/Impala tables (i.e. those with entries in the Hive Metastore) are authorized in the same way, regardless of whether backing storage is HDFS, S3, Kudu, HBase, etc - the SQL interface does the authorization to confirm that the end user has access to the table, columns, etc, then the service accesses the storage as the privileged user (Impala in this case).   In this model, if you create an external Kudu table in Impala and give permissions to a user to access the table via Impala, then they will have permissions to access the data in the underlying Kudu table. The thing that closes the loophole here is that creating the external Kudu table requires very high privileges - ALL permission on SERVER - a regular user can't create an external Kudu table pointed at an arbitrary Kudu cluster or table. ... View more

I lost this in my inbox but coming back.   GET_COLUMNS does use some of the same machinery as other queries but it's a metadata-only operation on metadata that can be entirely cached. Are you saying it consistently takes 500ms even if you run queries back-to-back? The only thing I can think of is if potentially you have a large number of databases or tables in your catalog. There is a step in the GET_COLUMNS processing where it searches through all the metadata to find something matching the tableName pattern in the request. ... View more

https://issues.apache.org/jira/browse/IMPALA-8454 is the apache impala jira ... View more

I don't have insight into how to solve your particular problem, but for what it's worth, in later versions of Impala (those included in CDP), Impala will read recursively from directories within the table location. ... View more

The client (i.e. the ODBC driver being used by your pyodbc program) is closing the insert operation before it finishes. I.e. it's starting the insert query, then closing the insert query before it's finished.   I don't know pyodbc well, but I wonder if it's something to do with how it's being used here. The examples I see either commit or fetch rows after execute().   I'd suggest trying either of those things (calling commit() or fetching from the cursor) to see if it forces your program to wait for the insert to succeed..   https://github.com/mkleehammer/pyodbc/wiki/Getting-started ... View more

The ODBC driver uses the column metadata to help implement some parts of the ODBC spec in my understanding.   The metadata used by the GET_COLUMNS operation should be cached in Impala's metadata cache, at least in most standard configurations that I can think of. The first GET_COLUMNS on a table could be quite slow, since it'll trigger loading all the table metadata, but after that it should be very fast - 500ms seems very slow for a table with cached metadata. Unless there was something like an "INVALIDATE METADATA" in-between.   Can you get a query profile for one of the GET_COLUMNS queries? That would have a timeline of how long the various steps took, like loading table metadata.   What version of Impala are you running? Have you got any non-standard configurations (like different catalog modes)? ... View more

I should also say - If you have a chance to upgrade your cluster, I think your experience with Impala would be improved quite a lot. The last CDH5 release - 5.16.2 is a big jump in scalability, performance and reliability from 5.10. CDH6.3.3 is a big jump beyond that in terms of features, then CDP is another huge step, particularly for metadata performance. ... View more

There's no dependency on any of the Cloudera management services.   Inserts are also going to depend on the HDFS service being healthy (i.e. namenodes, data nodes, etc).   There are various other underlying services that could be in play - Kerberos infrastructure like the KDC, the KMS if you're using certain encryption features, etc.   Those logs look like the client didn't actually close the query, so I'd question whether there was something that disrupted the client connect to the impala daemon (e.g. a load balancer was paused, or something happened to the client process).   ... View more