Member since
08-14-2018
3
Posts
0
Kudos Received
0
Solutions
10-19-2018
03:54 AM
We generated minimal sample data allowing to reproduce the issue. SQL to create a table: CREATE TABLE plat_test.test_users ( id INT, name STRING, devices ARRAY< STRUCT< id:STRING, device_info:STRUCT< model:STRING > > > ) STORED AS PARQUET insert some records and download the file via HUE interface
... View more
10-18-2018
06:21 AM
We got the Parquet file from HDFS created by CDH
... View more
08-14-2018
08:07 AM
Hi All, We found a case where Impala returns incorrect values from simple query. Our data contains nested array of structures and structures contains other structures. We generated minimal sample data allowing to reproduce the issue. SQL to create a table: CREATE TABLE plat_test.test_users ( id INT, name STRING, devices ARRAY< STRUCT< id:STRING, device_info:STRUCT< model:STRING > > > ) STORED AS PARQUET Please put attached parquet file to the location of the table and refresh the table. In sample data we have 2 users, one with 2 devices, second one with 3. Some of the devices.device_info.model fields are NULL. When I issue a query: SELECT u.name, d.device_info.model as model FROM test_users u, u.devices d; I'm expecting to get 5 records in results, but getting only one If I change query to: SELECT u.name, d.device_info.model as model FROM test_users u LEFT OUTER JOIN u.devices d; I'm getting two records in the results, but still not as it should be: We found some workaround to this problem. If we add to the result columns device.id we will get all records from parquet file: SELECT u.name, d.id, d.device_info.model as model FROM test_users u , u.devices d And result is But we can't rely on this workaround, because we don't need device.id in all queries and Impala optimizes it, and as a result we are getting unpredicted results. I tested Hive query on this table and it returns expected results: SELECT u.name, d.device_info.model FROM test_users u lateral view outer inline (u.devices) d; results: Please advice if it's a problem in Impala engine or we did some mistake in our query. test_users_131786401297925138_0.parquet Best regards, Come2Play team.
... View more
Labels:
- Labels:
-
Apache Impala