Reply
Highlighted
New Contributor
Posts: 3
Registered: ‎11-29-2018

JsonSerDe: SELECT * works, but field selection doesn't

Hopefully easy question, but this works:

 

select * from mytable; 

 

 

And this doesn't work:

select struct_col.field from mytable;

Error message:

 

java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found

 

 

This error doesn't make any sense to me because wouldn't the JsonSerDe have to be available to do the first select * statement?  Any tips on how to resolve this? Many thanks.

 

Using CDH 5.15.

 

 

The `show create table mytable` looks like

 

CREATE EXTERNAL TABLE `mytable`(                  |
|   `name` string COMMENT 'from deserializer',       |
|   `struct_col` struct<field:boolean,other:boolean,thing:boolean> COMMENT 'from deserializer')  |
| ROW FORMAT SERDE                                   |
|   'org.apache.hive.hcatalog.data.JsonSerDe'        |
| STORED AS INPUTFORMAT                              |
|   'org.apache.hadoop.mapred.TextInputFormat'       |
| OUTPUTFORMAT                                       |
|   'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' |
| LOCATION                                           |
|   'hdfs://host:8020/user/bob/mytable' |
| TBLPROPERTIES (                                    |
|   'COLUMN_STATS_ACCURATE'='false',                 |
|   'numFiles'='1',                                  |
|   'numRows'='-1',                                  |
|   'rawDataSize'='-1',                              |
|   'totalSize'='581',                               |
|   'transient_lastDdlTime'='1544062948')            |

 

 

 

New Contributor
Posts: 3
Registered: ‎11-29-2018

Re: JsonSerDe: SELECT * works, but field selection doesn't

I can't find the jar for this file anywhere.

 

org.apache.hive.hcatalog.data.JsonSerDe

 

Does anyone know where it is located? 

Cloudera Employee
Posts: 34
Registered: ‎08-16-2016

Re: JsonSerDe: SELECT * works, but field selection doesn't

The JsonSerDe class is in hive-hcatalog-xxx.jar.  So you will have to run "ADD JAR" on this file prior.

 

JsonSerDe has been made a first class citizen recently via

https://jira.apache.org/jira/browse/HIVE-18785.

With this fix you should be able to just use "STORED AS JSONFILE" instead of having to specify the ROW FORMAT SERDE

 

This fix is available in CDH6.1 ( I believe, dont quote me on this)

 

Hope this helps. Thanks

Announcements