Member since
11-29-2018
7
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4974 | 06-23-2019 07:33 AM |
01-02-2020
03:53 PM
1 Kudo
Yes looks like PIVOT would definitely work but that ticket appears to be 4 years old so I won't hold my breath on it 😉 In the mean time, I'll continue to use HIVE which provides this functionality with a simple dictionary like syntax (not quite PIVOT, but works and is intuitive for python programmers) `select name, props["age"], props["fav_animal"] from table;`
... View more
12-22-2019
10:46 AM
Hi,
In Impala, is it possible to project map keys from a MAP<string, string> as actual columns in the result set? I'm ingesting a dataset where we can't know all the possible attributes ahead of time and so we're using a map<string, string> column for maximum flexibility. However, at query time we still want to pull those attributes out as their own column.
Simplified example below with a few unique keys (in reality there are hundreds of keys in our dataset)
Schema:
Column
Type
name
STRING
props
MAP<string, string>
Raw Table:
Name
Props
Bob
{"age": "39", "fav_color": "green"}
Alice
{"fav_animal": "dog", "fav_color": "blue"}
Desired result:
name
age
fav_color
fav_animal
Bob
39
green
NULL
Alice
NULL
blue
dog
I can get all the results in a row-oriented manner by
select name, props.key, props.value from table, table.props props;
But, I've not been able to figure out how to make those keys from the MAP type column into their own columns. Is there a efficient way to do this in Impala?
... View more
Labels:
- Labels:
-
Apache Impala
06-23-2019
07:33 AM
This seems to work. select
snum, m.key, m.value
from table, table.items m;
... View more
06-21-2019
04:28 PM
Hi, I have a simple question about querying complex types. The schema, raw table and desired result is listed below. Could someone help me craft this query in Impala? CDH 5.15 I can get it to work in Hive, but it seems like Impala uses a join-like syntax to access complex columns. Many Thanks!! schema:
snum int
items map<string, float>
raw parquet table:
1, {"a": 1.3, "b", 9.1}
2, {"c", 84.9, "b", 1.1}
3, {}
4, {"x", 42.5,"p", 0.08}
desired result:
1,a,1.3
1,b,9.1
2,c,84.9
2,b,1.1
3,null,null
4,x,42.5
4,p,0.08
... View more
Labels:
- Labels:
-
Apache Impala
12-07-2018
04:12 PM
I can't find the jar for this file anywhere. org.apache.hive.hcatalog.data.JsonSerDe Does anyone know where it is located?
... View more
12-05-2018
08:23 PM
Hopefully easy question, but this works: select * from mytable; And this doesn't work: select struct_col.field from mytable; Error message: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found This error doesn't make any sense to me because wouldn't the JsonSerDe have to be available to do the first select * statement? Any tips on how to resolve this? Many thanks. Using CDH 5.15. The `show create table mytable` looks like CREATE EXTERNAL TABLE `mytable`( |
| `name` string COMMENT 'from deserializer', |
| `struct_col` struct<field:boolean,other:boolean,thing:boolean> COMMENT 'from deserializer') |
| ROW FORMAT SERDE |
| 'org.apache.hive.hcatalog.data.JsonSerDe' |
| STORED AS INPUTFORMAT |
| 'org.apache.hadoop.mapred.TextInputFormat' |
| OUTPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' |
| LOCATION |
| 'hdfs://host:8020/user/bob/mytable' |
| TBLPROPERTIES ( |
| 'COLUMN_STATS_ACCURATE'='false', |
| 'numFiles'='1', |
| 'numRows'='-1', |
| 'rawDataSize'='-1', |
| 'totalSize'='581', |
| 'transient_lastDdlTime'='1544062948') |
... View more
Labels:
- Labels:
-
Apache Hive